[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:23 dist-test-slave-cgv7 krb5kdc[11746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:23 dist-test-slave-cgv7 krb5kdc[11746](info): set up 2 sockets
Sep 29 18:40:23 dist-test-slave-cgv7 krb5kdc[11746](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 18:40:24.228263 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:24 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:24.361898 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33999
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:33999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:24.475531 11759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:24.475879 11759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:24.475953 11759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:24.480180 11759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:24.480279 11759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:24.480319 11759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:24.480372 11759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:24.480412 11759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:24.485906 11759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:33999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:24.487093 11759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:24.488145 11759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:24.497668 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:24.497736 11759 server_base.cc:1034] running on GCE node
W20260929 18:40:24.497668 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:24.497696 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:24.498651 11759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:24.499805 11759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:24.500969 11759 hybrid_clock.cc:648] HybridClock initialized: now 1790707224500939 us; error 44 us; skew 500 ppm
Sep 29 18:40:24 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707224, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:24.503826 11759 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:24.504693 11759 webserver.cc:460] Webserver started at http://127.11.118.254:33457/ using document root <none> and password file <none>
I20260929 18:40:24.505215 11759 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:24.505275 11759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:24.505439 11759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:24.507541 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "db707dc710e64079b441bd9909cbdecc"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "bb8cc2192f63cbfd3629ec828ed0ade5"
server_key_iv: "3311757f0162cb6b2e42d9f1d0743115"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.508054 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "db707dc710e64079b441bd9909cbdecc"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "bb8cc2192f63cbfd3629ec828ed0ade5"
server_key_iv: "3311757f0162cb6b2e42d9f1d0743115"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.512020 11759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:24.514647 11774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:24.516031 11759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:40:24.516224 11759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "db707dc710e64079b441bd9909cbdecc"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "bb8cc2192f63cbfd3629ec828ed0ade5"
server_key_iv: "3311757f0162cb6b2e42d9f1d0743115"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.516343 11759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:24.533471 11759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:24.536903 11759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:24.537124 11759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:24.544418 11759 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:33999
I20260929 18:40:24.544492 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:33999 every 8 connection(s)
I20260929 18:40:24.545841 11759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:24.548767 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 11759
I20260929 18:40:24.548897 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:24.548985 11827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:24.550580 11739 external_mini_cluster.cc:1428] Setting key 91a6e8330549e1d71c03c6a8a4fa87cf
I20260929 18:40:24.555840 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: Bootstrap starting.
I20260929 18:40:24.558203 11827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:24.559012 11827 log.cc:824] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:24.561491 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: No bootstrap required, opened a new log
I20260929 18:40:24.566605 11827 raft_consensus.cc:348] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } }
I20260929 18:40:24.566955 11827 raft_consensus.cc:374] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:24.567022 11827 raft_consensus.cc:729] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db707dc710e64079b441bd9909cbdecc, State: Initialized, Role: FOLLOWER
I20260929 18:40:24.567626 11827 consensus_queue.cc:260] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } }
I20260929 18:40:24.567865 11827 raft_consensus.cc:388] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:24.567952 11827 raft_consensus.cc:482] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:24.568087 11827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:24.569552 11827 raft_consensus.cc:504] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } }
I20260929 18:40:24.569999 11827 leader_election.cc:302] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db707dc710e64079b441bd9909cbdecc; no voters:
I20260929 18:40:24.570490 11827 leader_election.cc:288] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:24.570638 11832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 1 FOLLOWER]: Leader election won for term 1
Sep 29 18:40:24 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:24.570878 11832 raft_consensus.cc:686] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [term 1 LEADER]: Becoming Leader. State: Replica: db707dc710e64079b441bd9909cbdecc, State: Running, Role: LEADER
I20260929 18:40:24.571296 11832 consensus_queue.cc:237] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } }
I20260929 18:40:24.571882 11827 sys_catalog.cc:564] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:24.572815 11834 sys_catalog.cc:455] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader db707dc710e64079b441bd9909cbdecc. Latest consensus state: current_term: 1 leader_uuid: "db707dc710e64079b441bd9909cbdecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } } }
I20260929 18:40:24.573092 11834 sys_catalog.cc:458] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:24.573261 11833 sys_catalog.cc:455] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db707dc710e64079b441bd9909cbdecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db707dc710e64079b441bd9909cbdecc" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33999 } } }
I20260929 18:40:24.573527 11833 sys_catalog.cc:458] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:24.577469 11847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:24.577610 11847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:24.577688 11841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:24.578635 11841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:24.579106 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:24.553761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.11.118.254:33999)
0929 18:40:24.554239 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:24.554254 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:24.555468 (+ 1214us) server_negotiation.cc:407] Connection header received
0929 18:40:24.558636 (+ 3168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:24.558653 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:24.558969 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:24.559286 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:24.562396 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.563503 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.565604 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.566117 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.566345 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:24.572078 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:24.572086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:24.572104 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:24.572148 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:24.574637 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:24.575308 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:24.575315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:24.575321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:24.575503 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:24.576026 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:24.576030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:24.576031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:24.576571 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:24.576791 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:24.577594 (+ 803us) server_negotiation.cc:299] Negotiation successful
0929 18:40:24.577967 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":166,"threads_started":1}
I20260929 18:40:24.584774 11841 catalog_manager.cc:1347] Generated new cluster ID: 335058c08b6147e8b6f59bca01ddac33
I20260929 18:40:24.584867 11841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:24.610769 11841 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:24.611824 11841 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:24.617213 11841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db707dc710e64079b441bd9909cbdecc: Generated new TSK 0
I20260929 18:40:24.618114 11841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:24.648824 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:24.763325 11853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:24.763623 11853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:24.763671 11853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:24.767918 11853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:24.767998 11853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:24.768061 11853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:24.773712 11853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:24.774991 11853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:24.775975 11853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:24.785696 11859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:24.785696 11858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:24.785742 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:24.786000 11853 server_base.cc:1034] running on GCE node
I20260929 18:40:24.786484 11853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:24.787129 11853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:24.788313 11853 hybrid_clock.cc:648] HybridClock initialized: now 1790707224788297 us; error 27 us; skew 500 ppm
Sep 29 18:40:24 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707224, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:24.790999 11853 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:24.791857 11853 webserver.cc:460] Webserver started at http://127.11.118.193:40391/ using document root <none> and password file <none>
I20260929 18:40:24.792420 11853 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:24.792508 11853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:24.792733 11853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:24.794890 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "056ba877674341f49574054e07451862"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "d8b0389d920cd22d50b9c1a7b9bd9137"
server_key_iv: "f09a767eb9caa90c5168a4b17eafc898"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.795506 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "056ba877674341f49574054e07451862"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "d8b0389d920cd22d50b9c1a7b9bd9137"
server_key_iv: "f09a767eb9caa90c5168a4b17eafc898"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.799466 11853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:24.801847 11868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:24.803093 11853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:24.803260 11853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "056ba877674341f49574054e07451862"
format_stamp: "Formatted at 2026-09-29 18:40:24 on dist-test-slave-cgv7"
server_key: "d8b0389d920cd22d50b9c1a7b9bd9137"
server_key_iv: "f09a767eb9caa90c5168a4b17eafc898"
server_key_version: "encryptionkey@0"
I20260929 18:40:24.803385 11853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:24.827885 11853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:24.830837 11853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:24.831038 11853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:24.831619 11853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:24.832442 11853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:24.832515 11853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:24.832584 11853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:24.832628 11853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:24.841801 11853 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:38259
I20260929 18:40:24.841825 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:38259 every 8 connection(s)
I20260929 18:40:24.842764 11853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:24.845444 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 11853
I20260929 18:40:24.845570 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:24.845825 11739 external_mini_cluster.cc:1428] Setting key f29a12b7b826f8077a93eb8d9397bb1d
Sep 29 18:40:24 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707224, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:24.857853 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:24.844589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39843 (local address 127.11.118.254:33999)
0929 18:40:24.844740 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:24.844744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:24.845984 (+ 1240us) server_negotiation.cc:407] Connection header received
0929 18:40:24.847041 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:24.847044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:24.847108 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:24.847188 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:24.848400 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.849075 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.850262 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.850755 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.850881 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:24.853752 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:24.853758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:24.853764 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:24.853802 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:24.855694 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:24.856372 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:24.856376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:24.856377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:24.856457 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:24.856835 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:24.856837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:24.856838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:24.857021 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:24.857111 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:24.857617 (+ 506us) server_negotiation.cc:299] Negotiation successful
0929 18:40:24.857729 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:24.859045 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:24.845017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33999 (local address 127.11.118.193:39843)
0929 18:40:24.845804 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:24.845859 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:24.846832 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:24.847368 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:24.847375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:24.847755 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:24.848247 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:24.848260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.849214 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.849218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:24.850139 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:24.850149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:24.850917 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:24.850919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:24.851078 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:24.851730 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:24.851747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:24.853547 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:24.855872 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:24.855880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:24.855901 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:24.856249 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:24.856541 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:24.856543 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:24.856545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:24.856710 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:24.857118 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:24.857123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:24.857388 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 18:40:24.857554 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 18:40:24.857967 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":156,"threads_started":1}
I20260929 18:40:24.860535 11982 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33999
I20260929 18:40:24.860898 11982 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:24.861552 11982 heartbeater.cc:511] Master 127.11.118.254:33999 requested a full tablet report, sending...
I20260929 18:40:24.863364 11791 ts_manager.cc:194] Registered new tserver with Master: 056ba877674341f49574054e07451862 (127.11.118.193:38259)
I20260929 18:40:24.864532 11791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:39843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:24.878010 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:24.991487 11987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:24.991852 11987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:24.991927 11987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:24.996249 11987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:24.996353 11987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:24.996449 11987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:25.002167 11987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:25.003381 11987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:25.004482 11987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:25.014568 11992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:25.014707 11987 server_base.cc:1034] running on GCE node
W20260929 18:40:25.014596 11993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:25.014572 11995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:25.015350 11987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:25.016017 11987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:25.017202 11987 hybrid_clock.cc:648] HybridClock initialized: now 1790707225017166 us; error 50 us; skew 500 ppm
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:25.020591 11987 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:25.021720 11987 webserver.cc:460] Webserver started at http://127.11.118.194:43287/ using document root <none> and password file <none>
I20260929 18:40:25.022358 11987 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:25.022449 11987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:25.022666 11987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:25.024873 11987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "62eba3cebe3a4f7cbd1202f84427fd9e"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "e9a24068df9b8bacb7d92eb9aeef65f1"
server_key_iv: "ce44e127507e919a1c0eff7fdca0e170"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.025687 11987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "62eba3cebe3a4f7cbd1202f84427fd9e"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "e9a24068df9b8bacb7d92eb9aeef65f1"
server_key_iv: "ce44e127507e919a1c0eff7fdca0e170"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.029947 11987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 18:40:25.032657 12002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.034035 11987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:25.034216 11987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "62eba3cebe3a4f7cbd1202f84427fd9e"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "e9a24068df9b8bacb7d92eb9aeef65f1"
server_key_iv: "ce44e127507e919a1c0eff7fdca0e170"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.034396 11987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:25.066988 11987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:25.070003 11987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:25.070230 11987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:25.070870 11987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:25.071717 11987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:25.071784 11987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.071870 11987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:25.071938 11987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.081189 11987 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:33729
I20260929 18:40:25.081205 12115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:33729 every 8 connection(s)
I20260929 18:40:25.082109 11987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:25.084921 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 11987
I20260929 18:40:25.085036 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:25.085265 11739 external_mini_cluster.cc:1428] Setting key c3886a42f5b1a1869df3049384c54fdb
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:25.096370 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.084049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:58881 (local address 127.11.118.254:33999)
0929 18:40:25.084234 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:25.084239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:25.085421 (+ 1182us) server_negotiation.cc:407] Connection header received
0929 18:40:25.086382 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:25.086385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:25.086439 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:25.086518 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:25.087710 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.088230 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.089368 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.089603 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.089721 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.092663 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:25.092666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:25.092668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:25.092692 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:25.094435 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.094926 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.094929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.094931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.095008 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.095306 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.095308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.095309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.095553 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:25.095634 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:25.096130 (+ 496us) server_negotiation.cc:299] Negotiation successful
0929 18:40:25.096248 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:40:25.097213 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.084520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33999 (local address 127.11.118.194:58881)
0929 18:40:25.085247 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:25.085300 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:25.086123 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:25.086690 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:25.086697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:25.087064 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:25.087561 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:25.087574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.088358 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.088362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:25.089244 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:25.089253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.089764 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.089768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:25.089939 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.090595 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:25.090612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:25.092485 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:25.094557 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:25.094563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:25.094579 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:25.094832 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:25.095090 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:25.095092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:25.095093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:25.095227 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:25.095667 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:25.095672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:25.095921 (+ 249us) client_negotiation.cc:769] Sending connection context
0929 18:40:25.096086 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 18:40:25.096349 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:25.098567 12116 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33999
I20260929 18:40:25.098830 12116 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:25.099416 12116 heartbeater.cc:511] Master 127.11.118.254:33999 requested a full tablet report, sending...
I20260929 18:40:25.100586 11791 ts_manager.cc:194] Registered new tserver with Master: 62eba3cebe3a4f7cbd1202f84427fd9e (127.11.118.194:33729)
I20260929 18:40:25.101485 11791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:58881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:25.117076 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:25.232869 12121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:25.233183 12121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:25.233263 12121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:25.237370 12121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:25.237442 12121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:25.237557 12121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:25.243098 12121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33999
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:25.244161 12121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:25.245131 12121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:25.254691 12121 server_base.cc:1034] running on GCE node
W20260929 18:40:25.254662 12129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:25.254638 12126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:25.254638 12127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:25.255338 12121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:25.255971 12121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:25.257144 12121 hybrid_clock.cc:648] HybridClock initialized: now 1790707225257143 us; error 48 us; skew 500 ppm
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:25.259943 12121 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:25.260834 12121 webserver.cc:460] Webserver started at http://127.11.118.195:39997/ using document root <none> and password file <none>
I20260929 18:40:25.261389 12121 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:25.261447 12121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:25.261700 12121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:25.263818 12121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "65bd336bface4bc6901c3826a3ce1def"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "df4b53f02bcf898e034d3e734674b3e4"
server_key_iv: "835fcf096a2403f5959e7a0c56f339b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.264379 12121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "65bd336bface4bc6901c3826a3ce1def"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "df4b53f02bcf898e034d3e734674b3e4"
server_key_iv: "835fcf096a2403f5959e7a0c56f339b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.268522 12121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:25.271118 12136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.272475 12121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:40:25.272620 12121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "65bd336bface4bc6901c3826a3ce1def"
format_stamp: "Formatted at 2026-09-29 18:40:25 on dist-test-slave-cgv7"
server_key: "df4b53f02bcf898e034d3e734674b3e4"
server_key_iv: "835fcf096a2403f5959e7a0c56f339b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:25.272732 12121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:25.288398 12121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:25.292520 12121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:25.292692 12121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:25.293356 12121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:25.294261 12121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:25.294371 12121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.294420 12121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:25.294440 12121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:25.303684 12121 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:43211
I20260929 18:40:25.303721 12249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:43211 every 8 connection(s)
I20260929 18:40:25.304620 12121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:25.313764 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12121
I20260929 18:40:25.313867 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:25.314122 11739 external_mini_cluster.cc:1428] Setting key f56179da01e5a3a4296714596c5e99ce
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:25.318773 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.306654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:54783 (local address 127.11.118.254:33999)
0929 18:40:25.306805 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:25.306810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:25.307933 (+ 1123us) server_negotiation.cc:407] Connection header received
0929 18:40:25.308803 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:25.308806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:25.308853 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:25.308932 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:25.310106 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.310734 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.311863 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.312087 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.312185 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.315095 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:25.315099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:25.315106 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:25.315130 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:25.316678 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.317209 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.317214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.317216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.317296 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.317632 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.317637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.317638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.317826 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:25.317926 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:25.318494 (+ 568us) server_negotiation.cc:299] Negotiation successful
0929 18:40:25.318613 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:40:25.319597 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.307078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33999 (local address 127.11.118.195:54783)
0929 18:40:25.307793 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:25.307834 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:25.308608 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:25.309088 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:25.309095 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:25.309479 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:25.309960 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:25.309972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.310842 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.310845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:25.311729 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:25.311737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.312253 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.312256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:25.312401 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.313024 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:25.313041 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:25.314920 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:25.316809 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:25.316815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:25.316831 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:25.317108 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:25.317389 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:25.317393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:25.317395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:25.317541 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:25.317939 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:25.317947 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:25.318204 (+ 257us) client_negotiation.cc:769] Sending connection context
0929 18:40:25.318438 (+ 234us) client_negotiation.cc:240] Negotiation successful
0929 18:40:25.318776 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":137,"threads_started":1}
I20260929 18:40:25.320613 12250 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33999
I20260929 18:40:25.320852 12250 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:25.321350 12250 heartbeater.cc:511] Master 127.11.118.254:33999 requested a full tablet report, sending...
I20260929 18:40:25.322359 11791 ts_manager.cc:194] Registered new tserver with Master: 65bd336bface4bc6901c3826a3ce1def (127.11.118.195:43211)
I20260929 18:40:25.322924 11791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:54783
I20260929 18:40:25.328318 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:25.355944 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.346131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.11.118.193:38259)
0929 18:40:25.346444 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:25.346462 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:25.346489 (+ 27us) server_negotiation.cc:407] Connection header received
0929 18:40:25.346676 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:25.346683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:25.346892 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:25.347043 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:25.347781 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.348575 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.349433 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.349811 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.349928 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.352393 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:25.352403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:25.352423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:25.352448 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:25.354026 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.354542 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.354549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.354551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.354641 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.354922 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.354925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.354927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.355179 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:25.355422 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:25.355592 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 18:40:25.355752 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":70,"threads_started":1}
W20260929 18:40:25.356966 11896 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39088
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[11746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:25.383973 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:25.375141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.11.118.193:38259)
0929 18:40:25.375383 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:25.375388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:25.375527 (+ 139us) server_negotiation.cc:407] Connection header received
0929 18:40:25.375708 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:25.375711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:25.375754 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:25.375837 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:25.376575 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.377143 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.378003 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:25.378245 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:25.378422 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:25.381091 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:25.381094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:25.381096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:25.381123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:25.382473 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.382875 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.382877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.382879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.382949 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:25.383249 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:25.383252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:25.383253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:25.383429 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:25.383558 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:25.383748 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 18:40:25.383855 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260929 18:40:25.385015 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 11853
I20260929 18:40:25.390504 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 11987
I20260929 18:40:25.395593 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12121
I20260929 18:40:25.400697 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 11759
2026-09-29T18:40:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2236 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[12264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[12264](info): set up 2 sockets
Sep 29 18:40:25 dist-test-slave-cgv7 krb5kdc[12264](info): commencing operation
krb5kdc: starting...
W20260929 18:40:26.441850 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:26 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:26.569005 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41545
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:26.681502 12277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:26.681813 12277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:26.681900 12277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:26.687403 12277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:26.687482 12277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:26.687502 12277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:26.687520 12277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:26.687574 12277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:26.693250 12277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:26.694459 12277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:26.695521 12277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:26.704046 12283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:26.704052 12285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:26.704056 12282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:26.704209 12277 server_base.cc:1034] running on GCE node
I20260929 18:40:26.704926 12277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:26.706040 12277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:26.707235 12277 hybrid_clock.cc:648] HybridClock initialized: now 1790707226707183 us; error 68 us; skew 500 ppm
Sep 29 18:40:26 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:26.716183 12277 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:26.717242 12277 webserver.cc:460] Webserver started at http://127.11.118.254:37969/ using document root <none> and password file <none>
I20260929 18:40:26.717796 12277 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:26.717861 12277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:26.718035 12277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:26.720211 12277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "bc8988598a9b4c398be7122b5c36160b"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "05678381ea1ced24177aace02ad30f9a"
server_key_iv: "ab5953ea57d9ee17fa180ec35dc5d93a"
server_key_version: "encryptionkey@0"
I20260929 18:40:26.720745 12277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "bc8988598a9b4c398be7122b5c36160b"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "05678381ea1ced24177aace02ad30f9a"
server_key_iv: "ab5953ea57d9ee17fa180ec35dc5d93a"
server_key_version: "encryptionkey@0"
I20260929 18:40:26.724800 12277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:26.727329 12292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:26.728585 12277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:26.728724 12277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "bc8988598a9b4c398be7122b5c36160b"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "05678381ea1ced24177aace02ad30f9a"
server_key_iv: "ab5953ea57d9ee17fa180ec35dc5d93a"
server_key_version: "encryptionkey@0"
I20260929 18:40:26.728842 12277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:26.741895 12277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:26.745326 12277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:26.745579 12277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:26.752854 12277 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:41545
I20260929 18:40:26.752866 12344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:41545 every 8 connection(s)
I20260929 18:40:26.753844 12277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:26.755489 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12277
I20260929 18:40:26.755600 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:26.755863 11739 external_mini_cluster.cc:1428] Setting key 2f4da9abc036c70e3d5086ca00f925b0
I20260929 18:40:26.757167 12345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:26.763796 12345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b: Bootstrap starting.
Sep 29 18:40:26 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:26.766249 12345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:26.767203 12345 log.cc:824] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:26.769625 12345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b: No bootstrap required, opened a new log
I20260929 18:40:26.770671 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:26.757203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.11.118.254:41545)
0929 18:40:26.757632 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:26.757657 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:26.757692 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:40:26.758337 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:26.758354 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:26.758672 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:26.758955 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:26.759807 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:26.760806 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:26.761937 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:26.762463 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:26.762580 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:26.765399 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:26.765404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:26.765418 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:26.765447 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:26.767712 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:26.768206 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:26.768211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:26.768215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:26.768327 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:26.768626 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:26.768628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:26.768629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:26.769017 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:26.769153 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:26.769466 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 18:40:26.769828 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":141,"threads_started":1}
I20260929 18:40:26.773559 12345 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } }
I20260929 18:40:26.773795 12345 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:26.773877 12345 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8988598a9b4c398be7122b5c36160b, State: Initialized, Role: FOLLOWER
I20260929 18:40:26.774324 12345 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } }
I20260929 18:40:26.774478 12345 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:26.774557 12345 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:26.774653 12345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:26.775556 12345 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } }
I20260929 18:40:26.775916 12345 leader_election.cc:302] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc8988598a9b4c398be7122b5c36160b; no voters:
I20260929 18:40:26.776238 12345 leader_election.cc:288] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:26.776365 12350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:26.776616 12350 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [term 1 LEADER]: Becoming Leader. State: Replica: bc8988598a9b4c398be7122b5c36160b, State: Running, Role: LEADER
I20260929 18:40:26.776911 12350 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } }
I20260929 18:40:26.777406 12345 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:26.778584 12352 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc8988598a9b4c398be7122b5c36160b. Latest consensus state: current_term: 1 leader_uuid: "bc8988598a9b4c398be7122b5c36160b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } } }
I20260929 18:40:26.778579 12351 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc8988598a9b4c398be7122b5c36160b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc8988598a9b4c398be7122b5c36160b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41545 } } }
I20260929 18:40:26.778730 12352 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:26.778730 12351 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:26.779141 12359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:26.782323 12359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:26.787982 12359 catalog_manager.cc:1347] Generated new cluster ID: 6281cd62e01f45c28ca233d2b661b333
I20260929 18:40:26.788053 12359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:26.800114 12359 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:26.801024 12359 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:26.810479 12359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc8988598a9b4c398be7122b5c36160b: Generated new TSK 0
I20260929 18:40:26.811228 12359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:26.842101 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:26.958755 12371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:26.959034 12371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:26.959110 12371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:26.963300 12371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:26.963377 12371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:26.963474 12371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:26.969087 12371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:26.970227 12371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:26.971354 12371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:26.981335 12379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:26.981330 12376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:26.981415 12377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:26.981918 12371 server_base.cc:1034] running on GCE node
I20260929 18:40:26.982434 12371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:26.983124 12371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:26.984300 12371 hybrid_clock.cc:648] HybridClock initialized: now 1790707226984279 us; error 36 us; skew 500 ppm
Sep 29 18:40:26 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:26.987283 12371 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:26.988237 12371 webserver.cc:460] Webserver started at http://127.11.118.193:44433/ using document root <none> and password file <none>
I20260929 18:40:26.988842 12371 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:26.988914 12371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:26.989131 12371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:26.991372 12371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "832bd9a47dbf4b25b368524109f8e20d"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "92dbc08dc7cad3429b87b73b993f2e79"
server_key_iv: "a4a98255209e0369afff9eecd4fe15f2"
server_key_version: "encryptionkey@0"
I20260929 18:40:26.991947 12371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "832bd9a47dbf4b25b368524109f8e20d"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "92dbc08dc7cad3429b87b73b993f2e79"
server_key_iv: "a4a98255209e0369afff9eecd4fe15f2"
server_key_version: "encryptionkey@0"
I20260929 18:40:26.995802 12371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:26.998509 12386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:26.999768 12371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:26.999899 12371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "832bd9a47dbf4b25b368524109f8e20d"
format_stamp: "Formatted at 2026-09-29 18:40:26 on dist-test-slave-cgv7"
server_key: "92dbc08dc7cad3429b87b73b993f2e79"
server_key_iv: "a4a98255209e0369afff9eecd4fe15f2"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.000015 12371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:27.023559 12371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:27.027065 12371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:27.027309 12371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:27.027942 12371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:27.028944 12371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:27.029001 12371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.029050 12371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:27.029078 12371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.038882 12371 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:38771
I20260929 18:40:27.038898 12499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:38771 every 8 connection(s)
I20260929 18:40:27.039871 12371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:27.049095 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12371
I20260929 18:40:27.049228 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:27.049532 11739 external_mini_cluster.cc:1428] Setting key b8f1eaa7ede0f968b1ad9d11b3150453
Sep 29 18:40:27 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:27.054927 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.041786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:38153 (local address 127.11.118.254:41545)
0929 18:40:27.041937 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.041940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.043199 (+ 1259us) server_negotiation.cc:407] Connection header received
0929 18:40:27.044044 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.044047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.044102 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.044210 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:27.045472 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.046069 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.047330 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.047583 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.047673 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.050641 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:27.050645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:27.050648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:27.050672 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:27.052667 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.053238 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.053245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.053247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.053347 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.053689 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.053694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.053696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.053951 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:27.054064 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.054686 (+ 622us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.054800 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 18:40:27.056253 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.042209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41545 (local address 127.11.118.193:38153)
0929 18:40:27.042972 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.043027 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.043835 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.044371 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.044378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.044756 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:27.045311 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.045326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.046218 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.046221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.047197 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.047207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.047723 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.047726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.047868 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.048567 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:27.048584 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:27.050475 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:27.052798 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.052805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.052820 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.053117 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.053443 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.053445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.053446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.053588 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.054053 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:27.054058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:27.054404 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.054624 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.055026 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":136,"threads_started":1}
I20260929 18:40:27.057667 12500 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41545
I20260929 18:40:27.057955 12500 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:27.058629 12500 heartbeater.cc:511] Master 127.11.118.254:41545 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:27.060323 12309 ts_manager.cc:194] Registered new tserver with Master: 832bd9a47dbf4b25b368524109f8e20d (127.11.118.193:38771)
I20260929 18:40:27.061480 12309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:38153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:27.081214 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:27.194218 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:27.194545 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:27.194592 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:27.198791 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:27.198858 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:27.198968 12505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:27.204879 12505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:27.206012 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:27.207087 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:27.217435 12510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:27.217609 12505 server_base.cc:1034] running on GCE node
W20260929 18:40:27.217442 12511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:27.217442 12513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:27.218123 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:27.218871 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:27.220083 12505 hybrid_clock.cc:648] HybridClock initialized: now 1790707227220053 us; error 48 us; skew 500 ppm
Sep 29 18:40:27 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:27.223255 12505 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:27.224247 12505 webserver.cc:460] Webserver started at http://127.11.118.194:41457/ using document root <none> and password file <none>
I20260929 18:40:27.224882 12505 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:27.224985 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:27.225232 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:27.227757 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "b2b78e7246d447539cc7c1c39fb860f2"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "1ed53b26ede6ab100758174a60279bce"
server_key_iv: "2901cec0dbaff33587737c3fcb1e6d46"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.228407 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "b2b78e7246d447539cc7c1c39fb860f2"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "1ed53b26ede6ab100758174a60279bce"
server_key_iv: "2901cec0dbaff33587737c3fcb1e6d46"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.232870 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:27.235728 12520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.237242 12505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:27.237399 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "b2b78e7246d447539cc7c1c39fb860f2"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "1ed53b26ede6ab100758174a60279bce"
server_key_iv: "2901cec0dbaff33587737c3fcb1e6d46"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.237522 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:27.258034 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:27.261240 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:27.261451 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:27.262048 12505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:27.262930 12505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:27.262976 12505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.263041 12505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:27.263075 12505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.272145 12505 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:46833
I20260929 18:40:27.272145 12633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:46833 every 8 connection(s)
I20260929 18:40:27.273144 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:27.278172 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12505
I20260929 18:40:27.278337 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:27.278667 11739 external_mini_cluster.cc:1428] Setting key 34ff110cc7cc813a2d723d604a0db1e4
Sep 29 18:40:27 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:27.287421 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.275021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:55949 (local address 127.11.118.254:41545)
0929 18:40:27.275207 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.275212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.276102 (+ 890us) server_negotiation.cc:407] Connection header received
0929 18:40:27.276943 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.276946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.276993 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.277070 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:27.278663 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.279430 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.280680 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.280933 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.281046 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.283779 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:27.283782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:27.283785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:27.283809 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:27.285280 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.285820 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.285824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.285826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.285916 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.286257 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.286262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.286264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.286484 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:27.286592 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.287111 (+ 519us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.287252 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:40:27.288291 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.275478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41545 (local address 127.11.118.194:55949)
0929 18:40:27.275935 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.275978 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.276755 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.277250 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.277256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.277723 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:27.278470 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.278492 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.279548 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.279551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.280553 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.280563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.281089 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.281093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.281279 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.281903 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:27.281921 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:27.283593 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:27.285406 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.285412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.285430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.285712 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.286008 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.286012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.286014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.286162 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.286591 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:27.286599 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:27.286871 (+ 272us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.287067 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.287410 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
I20260929 18:40:27.289566 12634 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41545
I20260929 18:40:27.289844 12634 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:27.290532 12634 heartbeater.cc:511] Master 127.11.118.254:41545 requested a full tablet report, sending...
I20260929 18:40:27.291785 12309 ts_manager.cc:194] Registered new tserver with Master: b2b78e7246d447539cc7c1c39fb860f2 (127.11.118.194:46833)
I20260929 18:40:27.292433 12309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:55949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:27.306377 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:27.419755 12639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:27.420095 12639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:27.420176 12639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:27.424549 12639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:27.424652 12639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:27.424746 12639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:27.430349 12639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:27.431514 12639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:27.432663 12639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:27.443593 12645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:27.443717 12639 server_base.cc:1034] running on GCE node
W20260929 18:40:27.443614 12644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:27.443614 12647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:27.444370 12639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:27.445179 12639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:27.446445 12639 hybrid_clock.cc:648] HybridClock initialized: now 1790707227446401 us; error 59 us; skew 500 ppm
Sep 29 18:40:27 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:27.449707 12639 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:27.450834 12639 webserver.cc:460] Webserver started at http://127.11.118.195:40441/ using document root <none> and password file <none>
I20260929 18:40:27.451429 12639 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:27.451526 12639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:27.451758 12639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:27.453974 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "1aa952c8a0bf4b3e9304053d3738d3f8"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "5d091ffdada4f74dba50e953deb0f047"
server_key_iv: "82e9d3070db5b3f602ad8a0f84da6781"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.454594 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "1aa952c8a0bf4b3e9304053d3738d3f8"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "5d091ffdada4f74dba50e953deb0f047"
server_key_iv: "82e9d3070db5b3f602ad8a0f84da6781"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.458498 12639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:27.460973 12654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.462466 12639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:27.462602 12639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "1aa952c8a0bf4b3e9304053d3738d3f8"
format_stamp: "Formatted at 2026-09-29 18:40:27 on dist-test-slave-cgv7"
server_key: "5d091ffdada4f74dba50e953deb0f047"
server_key_iv: "82e9d3070db5b3f602ad8a0f84da6781"
server_key_version: "encryptionkey@0"
I20260929 18:40:27.462723 12639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:27.491392 12639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:27.494664 12639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:27.494872 12639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:27.495488 12639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:27.496335 12639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:27.496405 12639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.496474 12639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:27.496519 12639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:27.505769 12639 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:42207
I20260929 18:40:27.505785 12767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:42207 every 8 connection(s)
I20260929 18:40:27.506759 12639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:27.513253 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12639
I20260929 18:40:27.513427 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:27.513803 11739 external_mini_cluster.cc:1428] Setting key 772335d7878edd67907ac379f49ada6d
Sep 29 18:40:27 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:27.520663 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.508552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:57521 (local address 127.11.118.254:41545)
0929 18:40:27.508697 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.508701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.509663 (+ 962us) server_negotiation.cc:407] Connection header received
0929 18:40:27.510632 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.510635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.510682 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.510748 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:27.511909 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.512437 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.513705 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.514080 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.514208 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.517204 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:27.517207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:27.517209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:27.517234 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:27.518805 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.519295 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.519297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.519298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.519376 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.519672 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.519674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.519675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.519851 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:27.519925 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.520430 (+ 505us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.520541 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:27.521588 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.508853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41545 (local address 127.11.118.195:57521)
0929 18:40:27.509520 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.509561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.510419 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.510913 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.510920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.511311 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:27.511786 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.511798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.512547 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.512550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.513565 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.513576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.514310 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.514315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.514512 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.515305 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:27.515322 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:27.517040 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:27.518909 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.518915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.518928 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.519205 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.519452 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:27.519455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:27.519456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:27.519598 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:27.519931 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:27.519936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:27.520204 (+ 268us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.520365 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.520651 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":112,"threads_started":1}
I20260929 18:40:27.523056 12768 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41545
I20260929 18:40:27.523298 12768 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:27.523789 12768 heartbeater.cc:511] Master 127.11.118.254:41545 requested a full tablet report, sending...
I20260929 18:40:27.524844 12309 ts_manager.cc:194] Registered new tserver with Master: 1aa952c8a0bf4b3e9304053d3738d3f8 (127.11.118.195:42207)
I20260929 18:40:27.525468 12309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:57521
I20260929 18:40:27.528066 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:40:27.538830 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.531653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.11.118.254:41545)
0929 18:40:27.531772 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.531775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.531896 (+ 121us) server_negotiation.cc:407] Connection header received
0929 18:40:27.532005 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.532007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.532071 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.532153 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:27.532896 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.533475 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.534375 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.534587 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.534665 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.535927 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:27.535931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:27.535933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:27.535954 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:27.537348 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.537707 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.537709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.537710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.537789 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:27.538005 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:27.538007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:27.538008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:27.538260 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:27.538410 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.538616 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.538710 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:40:27.543772 12309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:27.546840 12309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:27.561951 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.555644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:46833 (local address 127.0.0.1:60960)
0929 18:40:27.556359 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.556414 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.556742 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.557414 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.557417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.557439 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.557674 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.557690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.558995 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.558998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.560514 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.560525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561389 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.561497 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.561543 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.561659 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.561789 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":186,"spinlock_wait_cycles":498176,"thread_start_us":122,"threads_started":1}
I20260929 18:40:27.561962 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.555827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.11.118.194:46833)
0929 18:40:27.556658 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.556672 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.556701 (+ 29us) server_negotiation.cc:407] Connection header received
0929 18:40:27.556895 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.556900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.557079 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.557238 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.557825 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.558867 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.560651 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561266 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561406 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.561548 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.561696 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.561793 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":112,"threads_started":1}
I20260929 18:40:27.562494 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.556286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:42207 (local address 127.0.0.1:36786)
0929 18:40:27.557354 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.557367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.557500 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.558170 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.558173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.558190 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.558283 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.558307 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.559474 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.559480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.560484 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.560518 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561787 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.561966 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.562016 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.562261 (+ 245us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.562358 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":80,"thread_start_us":79,"threads_started":1}
I20260929 18:40:27.562561 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.556500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.11.118.195:42207)
0929 18:40:27.557217 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.557229 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.557441 (+ 212us) server_negotiation.cc:407] Connection header received
0929 18:40:27.557684 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.557690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.557876 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.557987 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.558415 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.559348 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.560651 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561472 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561586 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.561702 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.562304 (+ 602us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.562415 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":161,"threads_started":1}
I20260929 18:40:27.562628 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.556019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.11.118.193:38771)
0929 18:40:27.556677 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.556691 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.556709 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:40:27.556788 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.556794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.557011 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.557191 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.557793 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.558774 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.560723 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561472 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561591 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.561702 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.562359 (+ 657us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.562461 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":70,"threads_started":1}
I20260929 18:40:27.562927 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.555516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:38771 (local address 127.0.0.1:47684)
0929 18:40:27.556358 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.556413 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.556584 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.557369 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.557376 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.557405 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.557669 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.557678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.558883 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.558888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.560634 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.560641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.561769 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.561773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.561963 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.562022 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.562091 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.562808 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":353,"thread_start_us":104,"threads_started":1}
I20260929 18:40:27.565578 12702 tablet_service.cc:1467] Processing CreateTablet for tablet a02c7edefd4347018bf53394dfb92d4b (DEFAULT_TABLE table=test-table [id=40dd483dfd3549ea870fc23a2fc561cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:27.565580 12568 tablet_service.cc:1467] Processing CreateTablet for tablet a02c7edefd4347018bf53394dfb92d4b (DEFAULT_TABLE table=test-table [id=40dd483dfd3549ea870fc23a2fc561cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:27.565579 12434 tablet_service.cc:1467] Processing CreateTablet for tablet a02c7edefd4347018bf53394dfb92d4b (DEFAULT_TABLE table=test-table [id=40dd483dfd3549ea870fc23a2fc561cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:27.566744 12702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a02c7edefd4347018bf53394dfb92d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:27.566738 12434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a02c7edefd4347018bf53394dfb92d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:27.566738 12568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a02c7edefd4347018bf53394dfb92d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:27.573846 12785 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Bootstrap starting.
I20260929 18:40:27.574666 12786 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Bootstrap starting.
I20260929 18:40:27.576409 12785 tablet_bootstrap.cc:654] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:27.576632 12786 tablet_bootstrap.cc:654] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:27.577430 12786 log.cc:824] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:27.577651 12785 log.cc:824] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:27.578285 12787 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Bootstrap starting.
I20260929 18:40:27.579773 12786 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: No bootstrap required, opened a new log
I20260929 18:40:27.579995 12786 ts_tablet_manager.cc:1397] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:40:27.580646 12785 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: No bootstrap required, opened a new log
I20260929 18:40:27.580860 12785 ts_tablet_manager.cc:1397] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 18:40:27.581073 12787 tablet_bootstrap.cc:654] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:27.581952 12787 log.cc:824] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:27.584026 12786 raft_consensus.cc:348] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.584272 12787 tablet_bootstrap.cc:492] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: No bootstrap required, opened a new log
I20260929 18:40:27.584299 12786 raft_consensus.cc:374] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:27.584350 12786 raft_consensus.cc:729] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832bd9a47dbf4b25b368524109f8e20d, State: Initialized, Role: FOLLOWER
I20260929 18:40:27.584527 12787 ts_tablet_manager.cc:1397] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260929 18:40:27.584849 12786 consensus_queue.cc:260] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.584761 12785 raft_consensus.cc:348] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.585057 12785 raft_consensus.cc:374] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:27.585135 12785 raft_consensus.cc:729] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa952c8a0bf4b3e9304053d3738d3f8, State: Initialized, Role: FOLLOWER
I20260929 18:40:27.585561 12785 consensus_queue.cc:260] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.585914 12786 ts_tablet_manager.cc:1428] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:40:27.586455 12500 heartbeater.cc:503] Master 127.11.118.254:41545 was elected leader, sending a full tablet report...
I20260929 18:40:27.586584 12785 ts_tablet_manager.cc:1428] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:40:27.586927 12768 heartbeater.cc:503] Master 127.11.118.254:41545 was elected leader, sending a full tablet report...
I20260929 18:40:27.588380 12787 raft_consensus.cc:348] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.588658 12787 raft_consensus.cc:374] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:27.588722 12787 raft_consensus.cc:729] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b78e7246d447539cc7c1c39fb860f2, State: Initialized, Role: FOLLOWER
I20260929 18:40:27.589200 12787 consensus_queue.cc:260] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.589850 12634 heartbeater.cc:503] Master 127.11.118.254:41545 was elected leader, sending a full tablet report...
I20260929 18:40:27.590184 12787 ts_tablet_manager.cc:1428] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
W20260929 18:40:27.757776 12769 tablet.cc:2369] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:40:27.774470 12635 tablet.cc:2369] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:40:27.791208 12501 tablet.cc:2369] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:27.969985 12791 raft_consensus.cc:482] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:27.970257 12791 raft_consensus.cc:504] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.971544 12791 leader_election.cc:288] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2b78e7246d447539cc7c1c39fb860f2 (127.11.118.194:46833), 1aa952c8a0bf4b3e9304053d3738d3f8 (127.11.118.195:42207)
I20260929 18:40:27.976635 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.972444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36941 (local address 127.11.118.195:42207)
0929 18:40:27.972623 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.972630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.972788 (+ 158us) server_negotiation.cc:407] Connection header received
0929 18:40:27.973016 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.973020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.973081 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.973169 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.973407 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.974067 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.975463 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.976100 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.976205 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.976251 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.976427 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.976499 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:40:27.976635 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.971970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:48267 (local address 127.11.118.194:46833)
0929 18:40:27.972227 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.972233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.972340 (+ 107us) server_negotiation.cc:407] Connection header received
0929 18:40:27.972518 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.972522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.972575 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.972658 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.973016 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.973811 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.975376 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.976022 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.976172 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.976225 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.976426 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.976499 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260929 18:40:27.976707 12794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.971852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:46833 (local address 127.11.118.193:48267)
0929 18:40:27.972227 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.972244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.972404 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.972698 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.972701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.972728 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.972876 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.972885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.973919 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.973922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.975221 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.975232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.976140 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.976161 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.976268 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.976300 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.976384 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.976491 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":2,"thread_start_us":121,"threads_started":1}
I20260929 18:40:27.976707 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.972327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:42207 (local address 127.11.118.193:36941)
0929 18:40:27.972683 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.972697 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.972804 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.973192 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.973194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.973209 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.973302 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.973306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.974157 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.974160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.975344 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.975351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.976217 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.976221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.976312 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.976338 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.976406 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.976495 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":26,"thread_start_us":94,"threads_started":1}
I20260929 18:40:27.977188 12792 raft_consensus.cc:482] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:27.977391 12792 raft_consensus.cc:504] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.977653 12588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "832bd9a47dbf4b25b368524109f8e20d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b78e7246d447539cc7c1c39fb860f2" is_pre_election: true
I20260929 18:40:27.977944 12588 raft_consensus.cc:2454] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 832bd9a47dbf4b25b368524109f8e20d in term 0.
I20260929 18:40:27.977967 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "832bd9a47dbf4b25b368524109f8e20d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" is_pre_election: true
I20260929 18:40:27.978168 12722 raft_consensus.cc:2454] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 832bd9a47dbf4b25b368524109f8e20d in term 0.
I20260929 18:40:27.978451 12389 leader_election.cc:302] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 832bd9a47dbf4b25b368524109f8e20d, b2b78e7246d447539cc7c1c39fb860f2; no voters:
I20260929 18:40:27.978742 12792 leader_election.cc:288] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 832bd9a47dbf4b25b368524109f8e20d (127.11.118.193:38771), b2b78e7246d447539cc7c1c39fb860f2 (127.11.118.194:46833)
I20260929 18:40:27.979005 12791 raft_consensus.cc:2781] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:27.979094 12791 raft_consensus.cc:482] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:27.979151 12791 raft_consensus.cc:3037] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:27.980598 12791 raft_consensus.cc:504] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.980979 12791 leader_election.cc:288] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [CANDIDATE]: Term 1 election: Requested vote from peers b2b78e7246d447539cc7c1c39fb860f2 (127.11.118.194:46833), 1aa952c8a0bf4b3e9304053d3738d3f8 (127.11.118.195:42207)
I20260929 18:40:27.981304 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "832bd9a47dbf4b25b368524109f8e20d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8"
I20260929 18:40:27.981423 12722 raft_consensus.cc:3037] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:27.982666 12722 raft_consensus.cc:2454] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 832bd9a47dbf4b25b368524109f8e20d in term 1.
I20260929 18:40:27.982977 12588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "832bd9a47dbf4b25b368524109f8e20d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b78e7246d447539cc7c1c39fb860f2"
I20260929 18:40:27.983029 12389 leader_election.cc:302] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa952c8a0bf4b3e9304053d3738d3f8, 832bd9a47dbf4b25b368524109f8e20d; no voters:
I20260929 18:40:27.983110 12588 raft_consensus.cc:3037] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:27.983799 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.979842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:42199 (local address 127.11.118.193:38771)
0929 18:40:27.979954 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.979958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.979967 (+ 9us) server_negotiation.cc:407] Connection header received
0929 18:40:27.979999 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.980001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.980059 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.980132 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.980363 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.981028 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.982287 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.982913 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.983032 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.983080 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.983608 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.983687 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:40:27.983798 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.979387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:46329 (local address 127.11.118.194:46833)
0929 18:40:27.979507 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:27.979511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:27.979573 (+ 62us) server_negotiation.cc:407] Connection header received
0929 18:40:27.979682 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:27.979684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:27.979719 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:27.979804 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:27.980070 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.980825 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.982365 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.982922 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.983031 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.983079 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:27.983617 (+ 538us) server_negotiation.cc:299] Negotiation successful
0929 18:40:27.983687 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:27.983798 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.978915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:38771 (local address 127.11.118.195:42199)
0929 18:40:27.979036 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.979054 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.979135 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.980141 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.980144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.980160 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.980259 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.980265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.981152 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.981157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.982122 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.982132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.983140 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.983144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.983386 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.983493 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.983575 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.983660 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":129}
I20260929 18:40:27.984239 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:27.979076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:46833 (local address 127.11.118.195:46329)
0929 18:40:27.979489 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:27.979505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:27.979603 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:27.979814 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:27.979816 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:27.979832 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:27.979961 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.979966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.980940 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.980943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.982188 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:27.982198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:27.983140 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:27.983144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:27.983388 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:27.983493 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 18:40:27.983967 (+ 474us) client_negotiation.cc:240] Negotiation successful
0929 18:40:27.984053 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":99,"thread_start_us":82,"threads_started":1}
I20260929 18:40:27.984351 12454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "832bd9a47dbf4b25b368524109f8e20d" is_pre_election: true
I20260929 18:40:27.984409 12588 raft_consensus.cc:2454] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 832bd9a47dbf4b25b368524109f8e20d in term 1.
I20260929 18:40:27.984611 12454 raft_consensus.cc:2379] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aa952c8a0bf4b3e9304053d3738d3f8 in current term 1: Already voted for candidate 832bd9a47dbf4b25b368524109f8e20d in this term.
I20260929 18:40:27.984810 12791 raft_consensus.cc:2781] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:27.985028 12791 raft_consensus.cc:686] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [term 1 LEADER]: Becoming Leader. State: Replica: 832bd9a47dbf4b25b368524109f8e20d, State: Running, Role: LEADER
I20260929 18:40:27.985316 12588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a02c7edefd4347018bf53394dfb92d4b" candidate_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b78e7246d447539cc7c1c39fb860f2" is_pre_election: true
I20260929 18:40:27.985399 12791 consensus_queue.cc:237] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } }
I20260929 18:40:27.985477 12588 raft_consensus.cc:2379] T a02c7edefd4347018bf53394dfb92d4b P b2b78e7246d447539cc7c1c39fb860f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aa952c8a0bf4b3e9304053d3738d3f8 in current term 1: Already voted for candidate 832bd9a47dbf4b25b368524109f8e20d in this term.
I20260929 18:40:27.985857 12657 leader_election.cc:302] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aa952c8a0bf4b3e9304053d3738d3f8; no voters: 832bd9a47dbf4b25b368524109f8e20d, b2b78e7246d447539cc7c1c39fb860f2
I20260929 18:40:27.986135 12792 raft_consensus.cc:2726] T a02c7edefd4347018bf53394dfb92d4b P 1aa952c8a0bf4b3e9304053d3738d3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 18:40:27.988603 12303 catalog_manager.cc:5495] T a02c7edefd4347018bf53394dfb92d4b P 832bd9a47dbf4b25b368524109f8e20d reported cstate change: term changed from 0 to 1, leader changed from <none> to 832bd9a47dbf4b25b368524109f8e20d (127.11.118.193). New cstate: current_term: 1 leader_uuid: "832bd9a47dbf4b25b368524109f8e20d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832bd9a47dbf4b25b368524109f8e20d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2b78e7246d447539cc7c1c39fb860f2" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 46833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa952c8a0bf4b3e9304053d3738d3f8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 42207 } health_report { overall_health: UNKNOWN } } }
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:28.017242 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:28.007748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.11.118.193:38771)
0929 18:40:28.007904 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:28.007910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:28.008061 (+ 151us) server_negotiation.cc:407] Connection header received
0929 18:40:28.008220 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:28.008224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:28.008275 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:28.008362 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:28.009077 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.009616 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.010536 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.010760 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.010870 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:28.013544 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:28.013556 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:28.013571 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:28.013599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:28.015418 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.015875 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.015881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.015882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.015971 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.016287 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.016291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.016293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.016552 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:28.016781 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:28.016940 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 18:40:28.017079 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:28.046581 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:28.037109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47704 (local address 127.11.118.193:38771)
0929 18:40:28.037261 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:28.037266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:28.037457 (+ 191us) server_negotiation.cc:407] Connection header received
0929 18:40:28.037572 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:28.037574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:28.037621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:28.037707 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:28.038397 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.039043 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.040002 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.040243 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.040355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:28.043178 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:28.043182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:28.043184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:28.043210 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:28.044763 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.045244 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.045252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.045254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.045331 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.045613 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.045617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.045619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.045796 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:28.045962 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:28.046260 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 18:40:28.046418 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:28.072871 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:28.064239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.11.118.193:38771)
0929 18:40:28.064426 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:28.064430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:28.064499 (+ 69us) server_negotiation.cc:407] Connection header received
0929 18:40:28.064636 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:28.064639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:28.064682 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:28.064752 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:28.065418 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.065908 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.066842 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:28.067093 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:28.067236 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:28.069751 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:28.069755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:28.069757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:28.069796 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:28.071359 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.071753 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.071755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.071757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.071828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:28.072072 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:28.072074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:28.072075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:28.072228 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:28.072328 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:28.072549 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 18:40:28.072657 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260929 18:40:28.074203 12414 tablet_service.cc:2908] Rejecting scan request for tablet a02c7edefd4347018bf53394dfb92d4b: Uninitialized: clean time has not yet been initialized
I20260929 18:40:28.076078 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12371
I20260929 18:40:28.081720 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12505
I20260929 18:40:28.087343 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12639
I20260929 18:40:28.092782 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12277
2026-09-29T18:40:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2692 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12811](info): set up 2 sockets
Sep 29 18:40:28 dist-test-slave-cgv7 krb5kdc[12811](info): commencing operation
krb5kdc: starting...
W20260929 18:40:29.141971 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:29.273085 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42021
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:42021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:29.389032 12824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:29.389374 12824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:29.389447 12824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:29.393656 12824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:29.393762 12824 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 18:40:29.393792 12824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:29.393836 12824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:29.393888 12824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:29.393927 12824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:29.399631 12824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:42021
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:29.400820 12824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:29.401885 12824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:29.411146 12832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:29.411201 12829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:29.411149 12830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:29.411424 12824 server_base.cc:1034] running on GCE node
I20260929 18:40:29.412107 12824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:29.413278 12824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:29.414458 12824 hybrid_clock.cc:648] HybridClock initialized: now 1790707229414435 us; error 39 us; skew 500 ppm
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:29.417357 12824 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:29.418370 12824 webserver.cc:460] Webserver started at http://127.11.118.254:39327/ using document root <none> and password file <none>
I20260929 18:40:29.418915 12824 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:29.418973 12824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:29.419152 12824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:29.421278 12824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "ba6e8231f3c942c4a96b038b2712571f"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "99a49cd0835fe6abf3fc499f9cb1c8ce"
server_key_iv: "6e75dfaed3515d522b9f98f8a8de18d9"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.421826 12824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "ba6e8231f3c942c4a96b038b2712571f"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "99a49cd0835fe6abf3fc499f9cb1c8ce"
server_key_iv: "6e75dfaed3515d522b9f98f8a8de18d9"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.426244 12824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 18:40:29.429171 12839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:29.430795 12824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:29.430917 12824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "ba6e8231f3c942c4a96b038b2712571f"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "99a49cd0835fe6abf3fc499f9cb1c8ce"
server_key_iv: "6e75dfaed3515d522b9f98f8a8de18d9"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.431149 12824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:29.452613 12824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:29.455797 12824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:29.455965 12824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:29.464165 12824 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:42021
I20260929 18:40:29.464210 12901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:42021 every 8 connection(s)
I20260929 18:40:29.465128 12824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:29.467914 12902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:29.469805 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12824
I20260929 18:40:29.469923 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:29.470194 11739 external_mini_cluster.cc:1428] Setting key b38eb6faa975cc81d9d663b5b69be2e4
I20260929 18:40:29.474632 12902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Bootstrap starting.
I20260929 18:40:29.476899 12902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:29.477650 12902 log.cc:824] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Log is configured to *not* fsync() on all Append() calls
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:29.479708 12902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: No bootstrap required, opened a new log
I20260929 18:40:29.483952 12902 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } }
I20260929 18:40:29.484230 12902 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:29.484277 12902 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6e8231f3c942c4a96b038b2712571f, State: Initialized, Role: FOLLOWER
I20260929 18:40:29.484694 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.471669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.11.118.254:42021)
0929 18:40:29.472136 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:29.472165 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:29.472207 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:40:29.472865 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:29.472884 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:29.473239 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:29.473583 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:29.474438 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.475465 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.476311 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.476791 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.476973 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:29.479419 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:29.479424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:29.479438 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:29.479465 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:29.481211 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.481719 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.481747 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.481753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.481902 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.482360 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.482364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.482365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.482921 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:29.483158 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:29.483370 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 18:40:29.483694 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":125,"threads_started":1}
I20260929 18:40:29.484766 12902 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } }
I20260929 18:40:29.484918 12902 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:29.484987 12902 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:29.485088 12902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:29.486238 12902 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } }
I20260929 18:40:29.486653 12902 leader_election.cc:302] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba6e8231f3c942c4a96b038b2712571f; no voters:
I20260929 18:40:29.487011 12902 leader_election.cc:288] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:29.487124 12907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:29.487418 12907 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [term 1 LEADER]: Becoming Leader. State: Replica: ba6e8231f3c942c4a96b038b2712571f, State: Running, Role: LEADER
I20260929 18:40:29.487766 12907 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } }
I20260929 18:40:29.488325 12902 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:29.489518 12908 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba6e8231f3c942c4a96b038b2712571f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } } }
I20260929 18:40:29.489549 12909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6e8231f3c942c4a96b038b2712571f. Latest consensus state: current_term: 1 leader_uuid: "ba6e8231f3c942c4a96b038b2712571f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6e8231f3c942c4a96b038b2712571f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42021 } } }
I20260929 18:40:29.489647 12909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:29.489650 12908 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:29.490121 12916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:29.492826 12916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:29.498122 12916 catalog_manager.cc:1347] Generated new cluster ID: c3756e091b0c43098f86e8acd51e68e1
I20260929 18:40:29.498199 12916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:29.536379 12916 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:29.537412 12916 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:29.550971 12916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Generated new TSK 0
I20260929 18:40:29.551725 12916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:29.583798 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42021
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:29.702886 12928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:29.703174 12928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:29.703260 12928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:29.707914 12928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:29.708024 12928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:29.708103 12928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:29.708182 12928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:29.713914 12928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:29.715080 12928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:29.716131 12928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:29.725885 12933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:29.725904 12936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:29.725994 12928 server_base.cc:1034] running on GCE node
W20260929 18:40:29.725904 12934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:29.726547 12928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:29.727275 12928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:29.728479 12928 hybrid_clock.cc:648] HybridClock initialized: now 1790707229728451 us; error 40 us; skew 500 ppm
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:29.731510 12928 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:29.732412 12928 webserver.cc:460] Webserver started at http://127.11.118.193:38413/ using document root <none> and password file <none>
I20260929 18:40:29.733021 12928 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:29.733114 12928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:29.733342 12928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:29.735797 12928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "5a31a9999be149f08e5602e4ab888c72"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "0af1f5fcdf0d9e01843e2942980870f0"
server_key_iv: "b99ace0445fcfa8628c5b217117b9b35"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.736382 12928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "5a31a9999be149f08e5602e4ab888c72"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "0af1f5fcdf0d9e01843e2942980870f0"
server_key_iv: "b99ace0445fcfa8628c5b217117b9b35"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.740227 12928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:29.742838 12943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:29.744166 12928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:29.744329 12928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "5a31a9999be149f08e5602e4ab888c72"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "0af1f5fcdf0d9e01843e2942980870f0"
server_key_iv: "b99ace0445fcfa8628c5b217117b9b35"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.744452 12928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:29.766923 12928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:29.770818 12928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:29.771045 12928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:29.772593 12928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:29.772702 12928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:29.772778 12928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:29.772819 12928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:29.784215 12928 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:34931
I20260929 18:40:29.784274 13058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:34931 every 8 connection(s)
I20260929 18:40:29.785436 12928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:29.787744 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.773664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:42271 (local address 127.11.118.254:42021)
0929 18:40:29.773793 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:29.773796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:29.774969 (+ 1173us) server_negotiation.cc:407] Connection header received
0929 18:40:29.776095 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:29.776099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:29.776157 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:29.776288 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:29.777678 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.778524 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.780144 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.780484 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.780602 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:29.783846 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:29.783849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:29.783852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:29.783881 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:29.785561 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.786192 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.786195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.786197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.786317 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.786668 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.786671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.786673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.786859 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:29.786963 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:29.787441 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 18:40:29.787571 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 18:40:29.788564 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.774258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.193:42271)
0929 18:40:29.774817 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:29.774861 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:29.775873 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:29.776462 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:29.776469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:29.776916 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:29.777525 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.777546 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.778646 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.778649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:29.779992 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.780005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.780644 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.780648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:29.780876 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:29.781568 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:29.781587 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:29.783636 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:29.785670 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:29.785675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:29.785687 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:29.786022 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:29.786413 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:29.786416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:29.786418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:29.786561 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:29.786959 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:29.786965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:29.787211 (+ 246us) client_negotiation.cc:769] Sending connection context
0929 18:40:29.787399 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 18:40:29.787719 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":162,"threads_started":1}
I20260929 18:40:29.789888 13059 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42021
I20260929 18:40:29.790061 13059 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:29.790629 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 12928
I20260929 18:40:29.790705 13059 heartbeater.cc:511] Master 127.11.118.254:42021 requested a full tablet report, sending...
I20260929 18:40:29.790760 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:29.791087 11739 external_mini_cluster.cc:1428] Setting key 20dbdfd6f527b42bae140368b2225ada
I20260929 18:40:29.792284 12856 ts_manager.cc:194] Registered new tserver with Master: 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193:34931)
I20260929 18:40:29.793913 12856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:42271
I20260929 18:40:29.800391 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.792148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.193:58153)
0929 18:40:29.792573 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:29.792587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:29.792703 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:29.792932 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:29.792935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:29.793145 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:29.793240 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.793246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.793978 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.793982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:29.794969 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.794983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.795721 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.795723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:29.795793 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:29.796299 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:29.796311 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:29.796650 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:29.798835 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:29.798838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:29.798840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:29.799053 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:29.799368 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:29.799370 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:29.799371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:29.799448 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:29.799898 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:29.799900 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:29.799989 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 18:40:29.800073 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:40:29.800225 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":193,"threads_started":1}
I20260929 18:40:29.800487 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.792198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:58153 (local address 127.11.118.254:42021)
0929 18:40:29.792302 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:29.792306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:29.792662 (+ 356us) server_negotiation.cc:407] Connection header received
0929 18:40:29.792779 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:29.792782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:29.792823 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:29.792909 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:29.793347 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.793858 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.795129 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.795509 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.795584 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:29.796799 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:29.796805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:29.796811 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:29.796842 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:29.798701 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.799157 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.799162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.799166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.799264 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:29.799535 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:29.799541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:29.799544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:29.799804 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:29.799920 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:29.800159 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 18:40:29.800299 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:29.822009 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42021
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:29.933816 13068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:29.934168 13068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:29.934245 13068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:29.938694 13068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:29.938777 13068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:29.938848 13068 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:29.938922 13068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:29.944389 13068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:29.945488 13068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:29.946627 13068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:29.956696 13073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:29.956699 13074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:29.956756 13068 server_base.cc:1034] running on GCE node
W20260929 18:40:29.956699 13076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:29.957428 13068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:29.958096 13068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:29.959275 13068 hybrid_clock.cc:648] HybridClock initialized: now 1790707229959248 us; error 46 us; skew 500 ppm
Sep 29 18:40:29 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:29.962045 13068 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:29.963009 13068 webserver.cc:460] Webserver started at http://127.11.118.194:36939/ using document root <none> and password file <none>
I20260929 18:40:29.963541 13068 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:29.963600 13068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:29.963763 13068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:29.965817 13068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "c2c3c3dd3d234f0e81a8f4eba8855942"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "7f137a531639c6c4172bb21ff88a85fe"
server_key_iv: "42f8178c60388c661caee7a9bd26e18c"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.966396 13068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "c2c3c3dd3d234f0e81a8f4eba8855942"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "7f137a531639c6c4172bb21ff88a85fe"
server_key_iv: "42f8178c60388c661caee7a9bd26e18c"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.970149 13068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:40:29.972594 13083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:29.973907 13068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:29.974040 13068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "c2c3c3dd3d234f0e81a8f4eba8855942"
format_stamp: "Formatted at 2026-09-29 18:40:29 on dist-test-slave-cgv7"
server_key: "7f137a531639c6c4172bb21ff88a85fe"
server_key_iv: "42f8178c60388c661caee7a9bd26e18c"
server_key_version: "encryptionkey@0"
I20260929 18:40:29.974134 13068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:29.987059 13068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:29.990561 13068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:29.990754 13068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:29.992106 13068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:29.992154 13068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:29.992216 13068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:29.992249 13068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:30.003324 13068 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:36833
I20260929 18:40:30.003389 13198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:36833 every 8 connection(s)
I20260929 18:40:30.004226 13068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:30.007258 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.992952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:60689 (local address 127.11.118.254:42021)
0929 18:40:29.993070 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:29.993074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:29.994179 (+ 1105us) server_negotiation.cc:407] Connection header received
0929 18:40:29.995394 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:29.995398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:29.995467 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:29.995585 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:29.997043 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.997868 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.999415 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.999855 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.999978 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.003231 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.003235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.003238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.003270 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.004979 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.005607 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.005610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.005611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.005700 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.006012 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.006015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.006017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.006213 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.006361 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.006979 (+ 618us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.007106 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 18:40:30.008055 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:29.993544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.194:60689)
0929 18:40:29.994011 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:29.994053 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:29.995114 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:29.995792 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:29.995802 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:29.996214 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:29.996903 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.996926 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:29.998037 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:29.998041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:29.999306 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:29.999316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.000021 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.000030 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.000216 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.000981 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:30.001002 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:30.003047 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:30.005150 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.005155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.005170 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.005490 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.005781 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.005783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.005784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.005916 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.006432 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:30.006438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:30.006703 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.006868 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.007138 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":133,"threads_started":1}
I20260929 18:40:30.008776 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13068
I20260929 18:40:30.008888 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:30.009145 11739 external_mini_cluster.cc:1428] Setting key 553950793c13ecee3d019835d2a0afd4
I20260929 18:40:30.009389 13199 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42021
I20260929 18:40:30.009564 13199 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:30.010313 13199 heartbeater.cc:511] Master 127.11.118.254:42021 requested a full tablet report, sending...
I20260929 18:40:30.011394 12856 ts_manager.cc:194] Registered new tserver with Master: c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833)
I20260929 18:40:30.012111 12856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:60689
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:30.020260 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.012278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:52139 (local address 127.11.118.254:42021)
0929 18:40:30.012406 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.012410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.012808 (+ 398us) server_negotiation.cc:407] Connection header received
0929 18:40:30.012943 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.012946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.012986 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.013070 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.013503 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.014386 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.015279 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.015696 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.015828 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.016900 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.016907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.016912 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.016943 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.018593 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.018974 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.018978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.018980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.019083 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.019352 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.019357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.019358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.019585 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.019710 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.019959 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.020070 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 18:40:30.020267 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.012464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.194:52139)
0929 18:40:30.012722 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.012737 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.012851 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.013085 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.013088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.013289 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:30.013386 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.013391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.014507 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.014509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.015163 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.015172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.015813 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.015815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.015882 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.016425 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:30.016444 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:30.016764 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:30.018695 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.018697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.018699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.018883 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.019179 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.019181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.019183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.019259 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.019707 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:30.019710 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:30.019802 (+ 92us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.019905 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.020052 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:30.038013 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42021
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:30.156613 13208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:30.156924 13208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:30.157019 13208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:30.161123 13208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:30.161223 13208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:30.161296 13208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:30.161381 13208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:30.166993 13208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:30.168120 13208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:30.169193 13208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:30.179077 13214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:30.179083 13213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:30.179116 13216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:30.179411 13208 server_base.cc:1034] running on GCE node
I20260929 18:40:30.179939 13208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:30.180696 13208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:30.181886 13208 hybrid_clock.cc:648] HybridClock initialized: now 1790707230181840 us; error 54 us; skew 500 ppm
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:30.184975 13208 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:30.185858 13208 webserver.cc:460] Webserver started at http://127.11.118.195:40085/ using document root <none> and password file <none>
I20260929 18:40:30.186462 13208 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:30.186529 13208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:30.186695 13208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:30.188714 13208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "5acbb931f92e4bc7a042cba2d53436e2"
format_stamp: "Formatted at 2026-09-29 18:40:30 on dist-test-slave-cgv7"
server_key: "ef306e0e150522f92b4e06adcf45508b"
server_key_iv: "1e452f8f509084856745b3baa8bdfd2f"
server_key_version: "encryptionkey@0"
I20260929 18:40:30.189291 13208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "5acbb931f92e4bc7a042cba2d53436e2"
format_stamp: "Formatted at 2026-09-29 18:40:30 on dist-test-slave-cgv7"
server_key: "ef306e0e150522f92b4e06adcf45508b"
server_key_iv: "1e452f8f509084856745b3baa8bdfd2f"
server_key_version: "encryptionkey@0"
I20260929 18:40:30.193430 13208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:40:30.195926 13223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:30.197227 13208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:40:30.197332 13208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "5acbb931f92e4bc7a042cba2d53436e2"
format_stamp: "Formatted at 2026-09-29 18:40:30 on dist-test-slave-cgv7"
server_key: "ef306e0e150522f92b4e06adcf45508b"
server_key_iv: "1e452f8f509084856745b3baa8bdfd2f"
server_key_version: "encryptionkey@0"
I20260929 18:40:30.197485 13208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:30.220302 13208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:30.223439 13208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:30.223668 13208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:30.225198 13208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:30.225246 13208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:30.225311 13208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:30.225343 13208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:30.236608 13208 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:43883
I20260929 18:40:30.236690 13338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:43883 every 8 connection(s)
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:30.237859 13208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:30.241649 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.226263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:51435 (local address 127.11.118.254:42021)
0929 18:40:30.226437 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.226443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.227392 (+ 949us) server_negotiation.cc:407] Connection header received
0929 18:40:30.228776 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.228781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.228845 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.228939 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.230877 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.231808 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.233808 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.234095 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.234303 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.237687 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.237691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.237693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.237723 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.239476 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.240023 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.240027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.240029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.240126 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.240501 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.240504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.240506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.240715 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.240824 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.241344 (+ 520us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.241475 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:40:30.242537 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.226690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.195:51435)
0929 18:40:30.227192 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.227255 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.228481 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.229161 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.229170 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.229741 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:30.230677 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.230702 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.232001 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.232007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.233640 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.233652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.234412 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.234420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.234677 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.235498 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:30.235520 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:30.237470 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:30.239613 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.239620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.239636 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.239904 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.240246 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.240249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.240251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.240397 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.240822 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:30.240830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:30.241090 (+ 260us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.241288 (+ 198us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.241623 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
I20260929 18:40:30.243732 13339 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42021
I20260929 18:40:30.243880 13339 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:30.244460 13339 heartbeater.cc:511] Master 127.11.118.254:42021 requested a full tablet report, sending...
I20260929 18:40:30.244855 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13208
I20260929 18:40:30.244979 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:30.245268 11739 external_mini_cluster.cc:1428] Setting key c51a44243f2f08d301642c87e56f7aa1
I20260929 18:40:30.246641 12855 ts_manager.cc:194] Registered new tserver with Master: 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.247442 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:40:30.247553 12855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:51435
I20260929 18:40:30.254964 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.246210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:49903 (local address 127.11.118.254:42021)
0929 18:40:30.246379 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.246384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.246674 (+ 290us) server_negotiation.cc:407] Connection header received
0929 18:40:30.246824 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.246827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.246875 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.246956 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.247448 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.248327 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.249762 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.249994 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.250105 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.251313 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.251317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.251320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.251346 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.253043 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.253502 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.253508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.253513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.253611 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.253945 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.253951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.253956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.254225 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.254375 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.254731 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.254834 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":85}
I20260929 18:40:30.255100 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.246215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.11.118.195:49903)
0929 18:40:30.246576 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.246590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.246696 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.246996 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.246999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.247230 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:30.247346 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.247353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.248522 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.248529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.249599 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.249610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.250111 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.250115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.250208 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.250801 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:30.250816 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:30.251166 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:30.253182 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.253185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.253186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.253395 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.253745 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.253747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.253749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.253841 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.254387 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:30.254392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:30.254535 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.254666 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.254850 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":135,"threads_started":1}
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:30.275959 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.267370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.11.118.254:42021)
0929 18:40:30.267544 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.267548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.267562 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:40:30.267605 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.267608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.267648 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.267726 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.268448 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.268999 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.269810 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.270042 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.270152 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.272866 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.272870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.272872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.272902 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.274523 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.274910 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.274912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.274914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.274985 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.275239 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.275241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.275243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.275426 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.275545 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.275736 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.275834 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 18:40:30.279219 12855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:30.282372 12855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:30.297654 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.292131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.11.118.193:34931)
0929 18:40:30.292462 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.292469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.292947 (+ 478us) server_negotiation.cc:407] Connection header received
0929 18:40:30.293040 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.293046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.293293 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.293478 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.293833 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.294910 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.296389 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.296961 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.297105 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.297257 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.297392 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.297490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":71,"threads_started":1}
I20260929 18:40:30.297646 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.291921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:34931 (local address 127.0.0.1:35354)
0929 18:40:30.292684 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.292707 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.292808 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.293483 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.293491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.293533 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.293684 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.293694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.295020 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.295026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.296267 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.296276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.297093 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.297097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.297204 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.297240 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.297364 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.297490 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"spinlock_wait_cycles":3328,"thread_start_us":94,"threads_started":1}
I20260929 18:40:30.298836 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.292894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:43883 (local address 127.0.0.1:34030)
0929 18:40:30.293439 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.293453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.293538 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.294431 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.294434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.294451 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.294552 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.294558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.295923 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.295926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.297065 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.297074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.298245 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.298249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.298471 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.298532 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.298609 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.298695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":97,"spinlock_wait_cycles":4224,"thread_start_us":102,"threads_started":1}
I20260929 18:40:30.298924 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.291972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.11.118.194:36833)
0929 18:40:30.293104 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.293678 (+ 574us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.293710 (+ 32us) server_negotiation.cc:407] Connection header received
0929 18:40:30.293788 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.293796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.293966 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.294240 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.294468 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.295607 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.297094 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.297919 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.298060 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.298179 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.298686 (+ 507us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.298781 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":133,"threads_started":1}
I20260929 18:40:30.299235 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.291803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36833 (local address 127.0.0.1:32844)
0929 18:40:30.292569 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.292653 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.292783 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.294208 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.294211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.294228 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.294362 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.294370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.295736 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.295740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.296952 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.296961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.298245 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.298249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.298468 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.298533 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.298607 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.299144 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":498,"thread_start_us":90,"threads_started":1}
I20260929 18:40:30.300057 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.293336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34030 (local address 127.11.118.195:43883)
0929 18:40:30.293798 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.293811 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.293831 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:40:30.293902 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.293907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.294121 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.294292 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.294677 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.295811 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.297185 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.297919 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.298067 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.298179 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.299791 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.299896 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":171,"threads_started":1}
I20260929 18:40:30.301438 12993 tablet_service.cc:1467] Processing CreateTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 (DEFAULT_TABLE table=test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.301821 13133 tablet_service.cc:1467] Processing CreateTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 (DEFAULT_TABLE table=test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.302690 12993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2734694bfe4d4ee98a68ba39452ffb64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.302842 13133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2734694bfe4d4ee98a68ba39452ffb64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.302888 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 (DEFAULT_TABLE table=test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.303872 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2734694bfe4d4ee98a68ba39452ffb64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.309799 13362 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Bootstrap starting.
I20260929 18:40:30.311195 13363 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Bootstrap starting.
I20260929 18:40:30.311527 13364 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Bootstrap starting.
I20260929 18:40:30.312466 13362 tablet_bootstrap.cc:654] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.313230 13363 tablet_bootstrap.cc:654] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.313369 13362 log.cc:824] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:30.313558 13364 tablet_bootstrap.cc:654] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.314105 13363 log.cc:824] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:30.314325 13364 log.cc:824] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:30.315809 13362 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: No bootstrap required, opened a new log
I20260929 18:40:30.315954 13363 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: No bootstrap required, opened a new log
I20260929 18:40:30.316020 13362 ts_tablet_manager.cc:1397] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:40:30.316111 13363 ts_tablet_manager.cc:1397] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:40:30.316145 13364 tablet_bootstrap.cc:492] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: No bootstrap required, opened a new log
I20260929 18:40:30.316339 13364 ts_tablet_manager.cc:1397] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:40:30.319231 13364 raft_consensus.cc:348] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.319507 13364 raft_consensus.cc:374] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.319562 13364 raft_consensus.cc:729] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acbb931f92e4bc7a042cba2d53436e2, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.320007 13364 consensus_queue.cc:260] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.320130 13362 raft_consensus.cc:348] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.320188 13363 raft_consensus.cc:348] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.320408 13363 raft_consensus.cc:374] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.320410 13362 raft_consensus.cc:374] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.320468 13363 raft_consensus.cc:729] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a31a9999be149f08e5602e4ab888c72, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.320468 13362 raft_consensus.cc:729] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c3c3dd3d234f0e81a8f4eba8855942, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.320828 13339 heartbeater.cc:503] Master 127.11.118.254:42021 was elected leader, sending a full tablet report...
I20260929 18:40:30.320904 13362 consensus_queue.cc:260] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.320904 13363 consensus_queue.cc:260] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.321127 13364 ts_tablet_manager.cc:1428] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:40:30.321822 13362 ts_tablet_manager.cc:1428] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:40:30.321822 13363 ts_tablet_manager.cc:1428] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:40:30.322149 13059 heartbeater.cc:503] Master 127.11.118.254:42021 was elected leader, sending a full tablet report...
I20260929 18:40:30.322149 13199 heartbeater.cc:503] Master 127.11.118.254:42021 was elected leader, sending a full tablet report...
I20260929 18:40:30.407999 13369 raft_consensus.cc:482] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:30.408245 13369 raft_consensus.cc:504] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.409674 13369 leader_election.cc:288] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.415582 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.410074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:52935 (local address 127.11.118.195:43883)
0929 18:40:30.410240 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.410244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.410557 (+ 313us) server_negotiation.cc:407] Connection header received
0929 18:40:30.410800 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.410803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.410858 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.410940 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.411475 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.412267 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.414320 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.414933 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.415067 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.415121 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.415364 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.415440 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:30.415710 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.410122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:41505 (local address 127.11.118.194:36833)
0929 18:40:30.410261 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.410288 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.410677 (+ 389us) server_negotiation.cc:407] Connection header received
0929 18:40:30.410867 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.410870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.410941 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.411013 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.411467 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.412317 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.414533 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.415154 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.415270 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.415347 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.415515 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.415584 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 18:40:30.415614 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.410008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:43883 (local address 127.11.118.193:52935)
0929 18:40:30.410434 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.410447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.410692 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.411069 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.411072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.411102 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.411332 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.411337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.412440 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.412443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.414163 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.414175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.415086 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.415089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.415181 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.415242 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.415335 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.415414 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":227,"spinlock_wait_cycles":768,"thread_start_us":139,"threads_started":1}
I20260929 18:40:30.416054 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.410010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36833 (local address 127.11.118.193:41505)
0929 18:40:30.410561 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.410577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.410707 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.411069 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.411072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.411106 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.411330 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.411337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.412440 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.412443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.414165 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.414175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.415287 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.415293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.415398 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.415417 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.415490 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.415544 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":272,"spinlock_wait_cycles":6272,"thread_start_us":143,"threads_started":1}
I20260929 18:40:30.416406 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2734694bfe4d4ee98a68ba39452ffb64" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2" is_pre_election: true
I20260929 18:40:30.416692 13293 raft_consensus.cc:2454] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 0.
I20260929 18:40:30.417306 12947 leader_election.cc:302] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a31a9999be149f08e5602e4ab888c72, 5acbb931f92e4bc7a042cba2d53436e2; no voters:
I20260929 18:40:30.417613 13369 raft_consensus.cc:2781] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:30.417694 13369 raft_consensus.cc:482] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:30.417726 13369 raft_consensus.cc:3037] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.417683 13153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2734694bfe4d4ee98a68ba39452ffb64" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" is_pre_election: true
I20260929 18:40:30.417991 13153 raft_consensus.cc:2454] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 0.
I20260929 18:40:30.418979 13369 raft_consensus.cc:504] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.419353 13369 leader_election.cc:288] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 election: Requested vote from peers c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.419669 13153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2734694bfe4d4ee98a68ba39452ffb64" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942"
I20260929 18:40:30.419692 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2734694bfe4d4ee98a68ba39452ffb64" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2"
I20260929 18:40:30.419795 13153 raft_consensus.cc:3037] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.419826 13293 raft_consensus.cc:3037] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.421134 13293 raft_consensus.cc:2454] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 1.
I20260929 18:40:30.421183 13153 raft_consensus.cc:2454] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 1.
I20260929 18:40:30.421480 12947 leader_election.cc:302] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a31a9999be149f08e5602e4ab888c72, 5acbb931f92e4bc7a042cba2d53436e2; no voters:
I20260929 18:40:30.421685 13369 raft_consensus.cc:2781] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:30.421887 13369 raft_consensus.cc:686] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 1 LEADER]: Becoming Leader. State: Replica: 5a31a9999be149f08e5602e4ab888c72, State: Running, Role: LEADER
I20260929 18:40:30.422238 13369 consensus_queue.cc:237] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.425566 12856 catalog_manager.cc:5495] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193). New cstate: current_term: 1 leader_uuid: "5a31a9999be149f08e5602e4ab888c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:30.459228 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.454841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.11.118.193:34931)
0929 18:40:30.454999 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.455004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.455018 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:40:30.455262 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.455264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.455321 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.455429 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:30.455722 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.456242 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.457140 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.457385 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.457500 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.457843 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.458334 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.458447 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.458944 (+ 497us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.459025 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:40:30.471129 13293 raft_consensus.cc:1261] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Refusing update from remote peer 5a31a9999be149f08e5602e4ab888c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.471222 13153 raft_consensus.cc:1261] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Refusing update from remote peer 5a31a9999be149f08e5602e4ab888c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.471886 13373 consensus_queue.cc:1035] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.472244 13369 consensus_queue.cc:1035] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.485253 13377 mvcc.cc:205] Tried to move back new op lower bound from 7334736816001261568 to 7334736815812706304. Current Snapshot: MvccSnapshot[applied={T|T < 7334736816001261568 or (T in {7334736816001261568})}]
I20260929 18:40:30.488138 13379 mvcc.cc:205] Tried to move back new op lower bound from 7334736816001261568 to 7334736815812706304. Current Snapshot: MvccSnapshot[applied={T|T < 7334736816001261568}]
I20260929 18:40:30.494606 12855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:30.494835 12855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:30.498116 12855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Sending DeleteTablet for 3 replicas of tablet 2734694bfe4d4ee98a68ba39452ffb64
I20260929 18:40:30.499074 13133 tablet_service.cc:1514] Processing DeleteTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32844
I20260929 18:40:30.499140 13273 tablet_service.cc:1514] Processing DeleteTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34030
I20260929 18:40:30.499135 12993 tablet_service.cc:1514] Processing DeleteTablet for tablet 2734694bfe4d4ee98a68ba39452ffb64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:35354
I20260929 18:40:30.499989 13386 tablet_replica.cc:330] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: stopping tablet replica
I20260929 18:40:30.500435 13386 raft_consensus.cc:2229] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:30.501161 13386 raft_consensus.cc:2258] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.501163 12855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:30.501730 12855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:30.503976 13386 ts_tablet_manager.cc:1905] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.508139 13133 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b (DEFAULT_TABLE table=test-table [id=7be224b40b2c446bb003489829a3383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.508183 12993 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b (DEFAULT_TABLE table=test-table [id=7be224b40b2c446bb003489829a3383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.508492 12993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed07b3124fb4f1e95ab14467bc5966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.508745 13388 tablet_replica.cc:330] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: stopping tablet replica
I20260929 18:40:30.508865 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b (DEFAULT_TABLE table=test-table [id=7be224b40b2c446bb003489829a3383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:30.509083 13388 raft_consensus.cc:2229] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:30.509140 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed07b3124fb4f1e95ab14467bc5966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.509447 13388 raft_consensus.cc:2258] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.510461 13387 tablet_replica.cc:330] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: stopping tablet replica
I20260929 18:40:30.510911 13387 raft_consensus.cc:2229] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:30.511209 13363 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Bootstrap starting.
I20260929 18:40:30.511255 13387 raft_consensus.cc:2258] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.512303 13363 tablet_bootstrap.cc:654] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.512998 13364 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Bootstrap starting.
I20260929 18:40:30.513586 13388 ts_tablet_manager.cc:1905] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.513784 13363 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: No bootstrap required, opened a new log
I20260929 18:40:30.513867 13363 ts_tablet_manager.cc:1397] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:30.513967 13364 tablet_bootstrap.cc:654] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.514415 13363 raft_consensus.cc:348] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.514524 13363 raft_consensus.cc:374] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.514554 13363 raft_consensus.cc:729] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a31a9999be149f08e5602e4ab888c72, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.514703 13363 consensus_queue.cc:260] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.514953 13363 ts_tablet_manager.cc:1428] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:30.515636 13387 ts_tablet_manager.cc:1905] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.518205 13387 ts_tablet_manager.cc:1918] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:30.518211 13388 ts_tablet_manager.cc:1918] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:30.518332 13388 log.cc:1196] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/2734694bfe4d4ee98a68ba39452ffb64
I20260929 18:40:30.518664 13388 ts_tablet_manager.cc:1939] T 2734694bfe4d4ee98a68ba39452ffb64 P 5a31a9999be149f08e5602e4ab888c72: Deleting consensus metadata
I20260929 18:40:30.519915 13133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed07b3124fb4f1e95ab14467bc5966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.519873 12841 catalog_manager.cc:4841] TS 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193:34931): tablet 2734694bfe4d4ee98a68ba39452ffb64 (table test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]) successfully deleted
I20260929 18:40:30.522433 13362 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Bootstrap starting.
I20260929 18:40:30.523456 13362 tablet_bootstrap.cc:654] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.524225 13364 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: No bootstrap required, opened a new log
I20260929 18:40:30.524335 13364 ts_tablet_manager.cc:1397] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260929 18:40:30.524963 13364 raft_consensus.cc:348] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.525098 13364 raft_consensus.cc:374] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.525138 13364 raft_consensus.cc:729] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acbb931f92e4bc7a042cba2d53436e2, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.525295 13364 consensus_queue.cc:260] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.525509 13386 ts_tablet_manager.cc:1918] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:30.525599 13386 log.cc:1196] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/2734694bfe4d4ee98a68ba39452ffb64
I20260929 18:40:30.525650 13364 ts_tablet_manager.cc:1428] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 18:40:30.525894 13386 ts_tablet_manager.cc:1939] T 2734694bfe4d4ee98a68ba39452ffb64 P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting consensus metadata
I20260929 18:40:30.527158 12842 catalog_manager.cc:4841] TS c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833): tablet 2734694bfe4d4ee98a68ba39452ffb64 (table test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]) successfully deleted
I20260929 18:40:30.530368 13387 log.cc:1196] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/2734694bfe4d4ee98a68ba39452ffb64
I20260929 18:40:30.530767 13387 ts_tablet_manager.cc:1939] T 2734694bfe4d4ee98a68ba39452ffb64 P 5acbb931f92e4bc7a042cba2d53436e2: Deleting consensus metadata
I20260929 18:40:30.531147 13362 tablet_bootstrap.cc:492] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: No bootstrap required, opened a new log
I20260929 18:40:30.531242 13362 ts_tablet_manager.cc:1397] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260929 18:40:30.531692 13362 raft_consensus.cc:348] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.531808 13362 raft_consensus.cc:374] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.531839 13362 raft_consensus.cc:729] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c3c3dd3d234f0e81a8f4eba8855942, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.531971 13362 consensus_queue.cc:260] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.532111 12843 catalog_manager.cc:4841] TS 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883): tablet 2734694bfe4d4ee98a68ba39452ffb64 (table test-table [id=b02293c69f5a4abdaec3b7b489f2ed7f]) successfully deleted
I20260929 18:40:30.532271 13362 ts_tablet_manager.cc:1428] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260929 18:40:30.536893 13060 tablet.cc:2369] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:30.593369 13382 raft_consensus.cc:482] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:30.593520 13382 raft_consensus.cc:504] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.593935 13382 leader_election.cc:288] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.594435 13153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed07b3124fb4f1e95ab14467bc5966b" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" is_pre_election: true
I20260929 18:40:30.594600 13153 raft_consensus.cc:2454] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 0.
I20260929 18:40:30.594919 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed07b3124fb4f1e95ab14467bc5966b" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2" is_pre_election: true
I20260929 18:40:30.594956 12946 leader_election.cc:302] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a31a9999be149f08e5602e4ab888c72, c2c3c3dd3d234f0e81a8f4eba8855942; no voters:
I20260929 18:40:30.595063 13293 raft_consensus.cc:2454] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 0.
I20260929 18:40:30.595352 13382 raft_consensus.cc:2781] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:30.595412 13382 raft_consensus.cc:482] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:30.595461 13382 raft_consensus.cc:3037] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.596699 13382 raft_consensus.cc:504] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.597074 13382 leader_election.cc:288] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 election: Requested vote from peers c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.597486 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed07b3124fb4f1e95ab14467bc5966b" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2"
I20260929 18:40:30.597608 13293 raft_consensus.cc:3037] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.598439 13153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed07b3124fb4f1e95ab14467bc5966b" candidate_uuid: "5a31a9999be149f08e5602e4ab888c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942"
I20260929 18:40:30.598593 13153 raft_consensus.cc:3037] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.598711 13293 raft_consensus.cc:2454] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 1.
I20260929 18:40:30.599005 12947 leader_election.cc:302] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a31a9999be149f08e5602e4ab888c72, 5acbb931f92e4bc7a042cba2d53436e2; no voters:
I20260929 18:40:30.599171 13382 raft_consensus.cc:2781] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:30.599238 13382 raft_consensus.cc:686] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 1 LEADER]: Becoming Leader. State: Replica: 5a31a9999be149f08e5602e4ab888c72, State: Running, Role: LEADER
I20260929 18:40:30.599380 13382 consensus_queue.cc:237] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.600049 13153 raft_consensus.cc:2454] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a31a9999be149f08e5602e4ab888c72 in term 1.
I20260929 18:40:30.601526 12856 catalog_manager.cc:5495] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193). New cstate: current_term: 1 leader_uuid: "5a31a9999be149f08e5602e4ab888c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } health_report { overall_health: UNKNOWN } } }
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:30.664589 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.652656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42021 (local address 127.0.0.1:55000)
0929 18:40:30.653117 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.653136 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.653287 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.653658 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.653663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.654322 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:30.654490 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.654501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.655627 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.655633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.656812 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.656827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.657433 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.657438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.657563 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.658302 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:30.658325 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:30.660316 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:30.662699 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.662709 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.662714 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.663011 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.663359 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:30.663362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:30.663364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:30.663459 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:30.663998 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:30.664005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:30.664132 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.664245 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.664398 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":120,"threads_started":1}
I20260929 18:40:30.665351 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.652749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.11.118.254:42021)
0929 18:40:30.652931 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.652937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.653243 (+ 306us) server_negotiation.cc:407] Connection header received
0929 18:40:30.653426 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.653429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.653508 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.653614 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.654649 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.655475 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.656956 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.657305 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.657407 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.660487 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.660493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.660497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.660532 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.662543 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.663120 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.663126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.663129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.663248 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.663566 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.663570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.663574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.663865 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.663995 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.664338 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.664506 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 18:40:30.670425 12856 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:55000:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 18:40:30.671196 12856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:30.676257 12993 tablet_service.cc:1467] Processing CreateTablet for tablet c12297ffa67a4f2c9baa7982d19f4555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f844ed8d6c4e4d7b9e5081c357ce06d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:30.676257 13133 tablet_service.cc:1467] Processing CreateTablet for tablet c12297ffa67a4f2c9baa7982d19f4555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f844ed8d6c4e4d7b9e5081c357ce06d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:30.676364 13273 tablet_service.cc:1467] Processing CreateTablet for tablet c12297ffa67a4f2c9baa7982d19f4555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f844ed8d6c4e4d7b9e5081c357ce06d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:30.676663 13133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12297ffa67a4f2c9baa7982d19f4555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.676674 12993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12297ffa67a4f2c9baa7982d19f4555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.676793 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12297ffa67a4f2c9baa7982d19f4555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:30.679520 13363 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: Bootstrap starting.
I20260929 18:40:30.679987 13364 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: Bootstrap starting.
I20260929 18:40:30.680466 13363 tablet_bootstrap.cc:654] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.680950 13364 tablet_bootstrap.cc:654] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.681104 13362 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: Bootstrap starting.
I20260929 18:40:30.681766 13363 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: No bootstrap required, opened a new log
I20260929 18:40:30.681872 13363 ts_tablet_manager.cc:1397] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:30.682101 13362 tablet_bootstrap.cc:654] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:30.682367 13363 raft_consensus.cc:348] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.682503 13363 raft_consensus.cc:374] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.682548 13363 raft_consensus.cc:729] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a31a9999be149f08e5602e4ab888c72, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.682673 13363 consensus_queue.cc:260] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.682996 13363 ts_tablet_manager.cc:1428] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:30.682955 13382 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.683125 13382 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.683446 13364 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: No bootstrap required, opened a new log
I20260929 18:40:30.683538 13364 ts_tablet_manager.cc:1397] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260929 18:40:30.684053 13364 raft_consensus.cc:348] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.684281 13364 raft_consensus.cc:374] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.684317 13362 tablet_bootstrap.cc:492] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: No bootstrap required, opened a new log
I20260929 18:40:30.684338 13364 raft_consensus.cc:729] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acbb931f92e4bc7a042cba2d53436e2, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.684413 13362 ts_tablet_manager.cc:1397] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 18:40:30.684502 13364 consensus_queue.cc:260] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.684785 13364 ts_tablet_manager.cc:1428] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:40:30.684875 13362 raft_consensus.cc:348] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.685071 13362 raft_consensus.cc:374] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:30.685141 13362 raft_consensus.cc:729] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c3c3dd3d234f0e81a8f4eba8855942, State: Initialized, Role: FOLLOWER
I20260929 18:40:30.685272 13362 consensus_queue.cc:260] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.685529 13362 ts_tablet_manager.cc:1428] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 18:40:30.685539 13370 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.685719 13370 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.685747 13368 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.685917 13368 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.715688 13370 raft_consensus.cc:482] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:30.715863 13370 raft_consensus.cc:504] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.717075 13370 leader_election.cc:288] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193:34931), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.721242 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.717445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:57005 (local address 127.11.118.195:43883)
0929 18:40:30.717567 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.717573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.717910 (+ 337us) server_negotiation.cc:407] Connection header received
0929 18:40:30.718058 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.718062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.718108 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.718212 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.718557 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.719044 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720211 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.720610 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720740 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.720799 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.721015 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.721086 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 18:40:30.721242 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.717600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:35735 (local address 127.11.118.193:34931)
0929 18:40:30.717706 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.717710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.717832 (+ 122us) server_negotiation.cc:407] Connection header received
0929 18:40:30.718039 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.718042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.718085 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.718185 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:30.718540 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.719046 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720211 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.720613 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720741 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.720800 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.721025 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.721102 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 18:40:30.721300 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.717352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:43883 (local address 127.11.118.194:57005)
0929 18:40:30.717828 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.717842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.717949 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.718225 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.718228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.718289 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.718442 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.718448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.719160 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.719166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.720073 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.720083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.720739 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.720875 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.720902 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.720998 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.721086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":191,"spinlock_wait_cycles":896,"thread_start_us":41,"threads_started":1}
I20260929 18:40:30.721305 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.717352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:34931 (local address 127.11.118.194:35735)
0929 18:40:30.717751 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.717769 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.717885 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.718194 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.718198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.718214 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:30.718414 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.718420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.719151 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.719158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.720082 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.720091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.720749 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.720753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.720892 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.720917 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.721006 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.721091 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":68,"spinlock_wait_cycles":21248,"thread_start_us":105,"threads_started":1}
I20260929 18:40:30.721787 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12297ffa67a4f2c9baa7982d19f4555" candidate_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2" is_pre_election: true
I20260929 18:40:30.721944 13293 raft_consensus.cc:2454] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2c3c3dd3d234f0e81a8f4eba8855942 in term 0.
I20260929 18:40:30.722088 13007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12297ffa67a4f2c9baa7982d19f4555" candidate_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a31a9999be149f08e5602e4ab888c72" is_pre_election: true
I20260929 18:40:30.722330 13087 leader_election.cc:302] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5acbb931f92e4bc7a042cba2d53436e2, c2c3c3dd3d234f0e81a8f4eba8855942; no voters:
I20260929 18:40:30.722371 13007 raft_consensus.cc:2454] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2c3c3dd3d234f0e81a8f4eba8855942 in term 0.
I20260929 18:40:30.722592 13370 raft_consensus.cc:2781] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:30.722658 13370 raft_consensus.cc:482] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:30.722708 13370 raft_consensus.cc:3037] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.723713 13370 raft_consensus.cc:504] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.724074 13370 leader_election.cc:288] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [CANDIDATE]: Term 1 election: Requested vote from peers 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193:34931), 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883)
I20260929 18:40:30.724488 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12297ffa67a4f2c9baa7982d19f4555" candidate_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbb931f92e4bc7a042cba2d53436e2"
I20260929 18:40:30.724601 13293 raft_consensus.cc:3037] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.725514 13293 raft_consensus.cc:2454] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2c3c3dd3d234f0e81a8f4eba8855942 in term 1.
I20260929 18:40:30.725809 13007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12297ffa67a4f2c9baa7982d19f4555" candidate_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a31a9999be149f08e5602e4ab888c72"
I20260929 18:40:30.725935 13007 raft_consensus.cc:3037] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:30.725975 13087 leader_election.cc:302] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5acbb931f92e4bc7a042cba2d53436e2, c2c3c3dd3d234f0e81a8f4eba8855942; no voters:
I20260929 18:40:30.726221 13370 raft_consensus.cc:2781] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:30.726473 13370 raft_consensus.cc:686] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 LEADER]: Becoming Leader. State: Replica: c2c3c3dd3d234f0e81a8f4eba8855942, State: Running, Role: LEADER
I20260929 18:40:30.726738 13370 consensus_queue.cc:237] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } }
I20260929 18:40:30.727123 13007 raft_consensus.cc:2454] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2c3c3dd3d234f0e81a8f4eba8855942 in term 1.
I20260929 18:40:30.728874 13404 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: TxnStatusTablet state changed. Reason: New leader c2c3c3dd3d234f0e81a8f4eba8855942. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.728971 13404 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:30.729198 13406 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 18:40:30.729187 12855 catalog_manager.cc:5495] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:30.729359 13406 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260929 18:40:30.740945 13340 tablet.cc:2369] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:30.749392 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.745202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.11.118.194:36833)
0929 18:40:30.745329 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.745333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.745346 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:40:30.745529 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.745532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.745589 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.745677 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:30.745942 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.746554 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.747766 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.748041 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.748135 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.748465 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.748952 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.749090 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.749171 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.749221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 18:40:30.749402 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.745057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36833 (local address 127.0.0.1:32846)
0929 18:40:30.745230 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.745246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.745351 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.745710 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.745713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.745720 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:30.745829 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.745835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.746656 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.746659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.747648 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.747658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.748149 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.748153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.748272 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.748332 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.749069 (+ 737us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.749085 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.749162 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.749257 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 18:40:30.752673 13293 raft_consensus.cc:1261] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Refusing update from remote peer c2c3c3dd3d234f0e81a8f4eba8855942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.752774 13007 raft_consensus.cc:1261] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Refusing update from remote peer c2c3c3dd3d234f0e81a8f4eba8855942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.753288 13370 consensus_queue.cc:1035] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.753521 13404 consensus_queue.cc:1035] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.755000 13368 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: TxnStatusTablet state changed. Reason: New leader c2c3c3dd3d234f0e81a8f4eba8855942. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.755136 13368 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.755323 13382 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: TxnStatusTablet state changed. Reason: New leader c2c3c3dd3d234f0e81a8f4eba8855942. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.755429 13382 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: This TxnStatusTablet replica's current role is: FOLLOWER
W20260929 18:40:30.755520 13200 tablet.cc:2369] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:30.757916 13370 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.758041 13370 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:30.758149 13370 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.758212 13370 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P c2c3c3dd3d234f0e81a8f4eba8855942: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:30.758704 13376 mvcc.cc:205] Tried to move back new op lower bound from 7334736817157173248 to 7334736817059479552. Current Snapshot: MvccSnapshot[applied={T|T < 7334736817157173248}]
I20260929 18:40:30.758869 13382 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.758957 13382 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5a31a9999be149f08e5602e4ab888c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.761350 13368 tablet_replica.cc:439] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a31a9999be149f08e5602e4ab888c72" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 34931 } } peers { permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 } } peers { permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 } } }
I20260929 18:40:30.761463 13368 tablet_replica.cc:442] T c12297ffa67a4f2c9baa7982d19f4555 P 5acbb931f92e4bc7a042cba2d53436e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:30.773168 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.769655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36969 (local address 127.11.118.194:36833)
0929 18:40:30.769815 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.769819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.769907 (+ 88us) server_negotiation.cc:407] Connection header received
0929 18:40:30.770042 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.770044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.770085 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.770146 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:30.770409 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.770893 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.772137 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.772345 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.772434 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.772754 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.772828 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.772885 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.773015 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.773061 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260929 18:40:30.773239 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.769508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36833 (local address 127.11.118.193:36969)
0929 18:40:30.769837 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.769852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.769960 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.770179 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.770182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.770191 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:30.770323 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.770330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.771000 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.771004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.772010 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.772021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.772462 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.772466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.772564 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.772618 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.772919 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.772926 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.772993 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.773081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":124,"threads_started":1}
I20260929 18:40:30.779618 13153 raft_consensus.cc:1261] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Refusing update from remote peer 5a31a9999be149f08e5602e4ab888c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.779980 13293 raft_consensus.cc:1261] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Refusing update from remote peer 5a31a9999be149f08e5602e4ab888c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:30.780047 13373 consensus_queue.cc:1035] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c3c3dd3d234f0e81a8f4eba8855942" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.780365 13373 consensus_queue.cc:1035] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acbb931f92e4bc7a042cba2d53436e2" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:30.812307 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.808152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:36977 (local address 127.11.118.193:34931)
0929 18:40:30.808289 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.808294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.808462 (+ 168us) server_negotiation.cc:407] Connection header received
0929 18:40:30.808560 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.808563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.808615 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.808695 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:30.808917 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.809760 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.811059 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.811449 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.811536 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.811848 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.811929 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.812033 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.812150 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.812202 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:30.812404 13418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.808006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:34931 (local address 127.11.118.194:36977)
0929 18:40:30.808384 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:30.808397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:30.808485 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:30.808710 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:30.808712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:30.808718 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:30.808823 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.808827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.809888 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.809891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.810929 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:30.810940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.811545 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.811549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:30.811653 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.811711 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:30.812058 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:30.812065 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 18:40:30.812131 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 18:40:30.812215 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":147,"threads_started":1}
I20260929 18:40:30.841262 12853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:30.841445 12853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:30.843660 12853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba6e8231f3c942c4a96b038b2712571f: Sending DeleteTablet for 3 replicas of tablet 6ed07b3124fb4f1e95ab14467bc5966b
I20260929 18:40:30.844322 13133 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32844
I20260929 18:40:30.844398 12993 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:35354
I20260929 18:40:30.844406 13273 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed07b3124fb4f1e95ab14467bc5966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:30 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34030
I20260929 18:40:30.844586 13388 tablet_replica.cc:330] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: stopping tablet replica
I20260929 18:40:30.844583 13387 tablet_replica.cc:330] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: stopping tablet replica
I20260929 18:40:30.844637 13386 tablet_replica.cc:330] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: stopping tablet replica
I20260929 18:40:30.844703 13387 raft_consensus.cc:2229] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:30.844709 13388 raft_consensus.cc:2229] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:30.844748 13386 raft_consensus.cc:2229] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:30.844823 13387 raft_consensus.cc:2258] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.844843 13386 raft_consensus.cc:2258] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.844910 13388 raft_consensus.cc:2258] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:30.845676 13386 ts_tablet_manager.cc:1905] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.845705 13388 ts_tablet_manager.cc:1905] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.848433 13386 ts_tablet_manager.cc:1918] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:30.848532 13386 log.cc:1196] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/6ed07b3124fb4f1e95ab14467bc5966b
I20260929 18:40:30.848794 13386 ts_tablet_manager.cc:1939] T 6ed07b3124fb4f1e95ab14467bc5966b P c2c3c3dd3d234f0e81a8f4eba8855942: Deleting consensus metadata
I20260929 18:40:30.849107 13388 ts_tablet_manager.cc:1918] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:30.849188 13388 log.cc:1196] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/6ed07b3124fb4f1e95ab14467bc5966b
I20260929 18:40:30.849438 13388 ts_tablet_manager.cc:1939] T 6ed07b3124fb4f1e95ab14467bc5966b P 5a31a9999be149f08e5602e4ab888c72: Deleting consensus metadata
I20260929 18:40:30.849709 12842 catalog_manager.cc:4841] TS c2c3c3dd3d234f0e81a8f4eba8855942 (127.11.118.194:36833): tablet 6ed07b3124fb4f1e95ab14467bc5966b (table test-table [id=7be224b40b2c446bb003489829a3383f]) successfully deleted
I20260929 18:40:30.849852 13387 ts_tablet_manager.cc:1905] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:30.850297 12841 catalog_manager.cc:4841] TS 5a31a9999be149f08e5602e4ab888c72 (127.11.118.193:34931): tablet 6ed07b3124fb4f1e95ab14467bc5966b (table test-table [id=7be224b40b2c446bb003489829a3383f]) successfully deleted
I20260929 18:40:30.852666 13387 ts_tablet_manager.cc:1918] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:30.852766 13387 log.cc:1196] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6ed07b3124fb4f1e95ab14467bc5966b
I20260929 18:40:30.853060 13387 ts_tablet_manager.cc:1939] T 6ed07b3124fb4f1e95ab14467bc5966b P 5acbb931f92e4bc7a042cba2d53436e2: Deleting consensus metadata
I20260929 18:40:30.853847 12843 catalog_manager.cc:4841] TS 5acbb931f92e4bc7a042cba2d53436e2 (127.11.118.195:43883): tablet 6ed07b3124fb4f1e95ab14467bc5966b (table test-table [id=7be224b40b2c446bb003489829a3383f]) successfully deleted
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[12811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:30.861325 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.849369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.11.118.193:34931)
0929 18:40:30.849504 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.849508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.850497 (+ 989us) server_negotiation.cc:407] Connection header received
0929 18:40:30.850686 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.850688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.850740 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.850788 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.851819 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.852529 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.853744 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.854075 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.854376 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.857592 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.857605 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.857617 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.857646 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.859475 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.859940 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.859946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.859947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.860034 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.860331 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.860336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.860337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.860593 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.860807 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.861014 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.861147 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260929 18:40:30.861774 13053 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35376
I20260929 18:40:30.872071 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:30.863626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.11.118.254:42021)
0929 18:40:30.863761 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:30.863768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:30.863837 (+ 69us) server_negotiation.cc:407] Connection header received
0929 18:40:30.863959 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:30.863961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:30.864005 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:30.864072 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:30.864880 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.865463 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.866650 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:30.866875 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:30.866955 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:30.868623 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:30.868627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:30.868629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:30.868654 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:30.870158 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.870729 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.870732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.870734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.870813 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:30.871120 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:30.871122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:30.871123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:30.871314 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:30.871413 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:30.871754 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 18:40:30.871866 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 18:40:30.872498 12853 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55016
I20260929 18:40:30.874423 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12928
I20260929 18:40:30.884855 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13068
I20260929 18:40:30.892046 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13208
I20260929 18:40:30.897778 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 12824
2026-09-29T18:40:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2805 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[13427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[13427](info): set up 2 sockets
Sep 29 18:40:30 dist-test-slave-cgv7 krb5kdc[13427](info): commencing operation
krb5kdc: starting...
W20260929 18:40:31.954138 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:32.081866 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39097
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:32.192342 13440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:32.192667 13440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:32.192740 13440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:32.196949 13440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:32.197055 13440 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 18:40:32.197088 13440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:32.197122 13440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:32.197182 13440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:32.197219 13440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:32.202726 13440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39097
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:32.203919 13440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:32.204984 13440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:32.213948 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.214028 13448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.214076 13440 server_base.cc:1034] running on GCE node
W20260929 18:40:32.214303 13445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.215179 13440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:32.216322 13440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:32.217478 13440 hybrid_clock.cc:648] HybridClock initialized: now 1790707232217460 us; error 32 us; skew 500 ppm
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:32.223929 13440 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:32.224874 13440 webserver.cc:460] Webserver started at http://127.11.118.254:44221/ using document root <none> and password file <none>
I20260929 18:40:32.225497 13440 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:32.225586 13440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:32.225806 13440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:32.227909 13440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "a37efa6e3c5549dfa532fc2feb30dcc4"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "3d2c6f31cd82eb0f4cb021eaa31e3240"
server_key_iv: "f0091ec19ec04b14e485e4e026f68223"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.228489 13440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "a37efa6e3c5549dfa532fc2feb30dcc4"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "3d2c6f31cd82eb0f4cb021eaa31e3240"
server_key_iv: "f0091ec19ec04b14e485e4e026f68223"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.232295 13440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:32.234889 13455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.236267 13440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:32.236418 13440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "a37efa6e3c5549dfa532fc2feb30dcc4"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "3d2c6f31cd82eb0f4cb021eaa31e3240"
server_key_iv: "f0091ec19ec04b14e485e4e026f68223"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.236544 13440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:32.246822 13440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:32.250030 13440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:32.250247 13440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:32.257943 13440 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39097
I20260929 18:40:32.258041 13517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39097 every 8 connection(s)
I20260929 18:40:32.258996 13440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:32.261741 13518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:32.267293 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Bootstrap starting.
I20260929 18:40:32.268392 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13440
I20260929 18:40:32.268525 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:32.268801 11739 external_mini_cluster.cc:1428] Setting key 1706451be7a8c125669a0bc08934186a
I20260929 18:40:32.270305 13518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:32.271062 13518 log.cc:824] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:32.273383 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: No bootstrap required, opened a new log
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:32.278033 13518 raft_consensus.cc:348] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } }
I20260929 18:40:32.278332 13518 raft_consensus.cc:374] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:32.278419 13518 raft_consensus.cc:729] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37efa6e3c5549dfa532fc2feb30dcc4, State: Initialized, Role: FOLLOWER
I20260929 18:40:32.278939 13518 consensus_queue.cc:260] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } }
I20260929 18:40:32.279112 13518 raft_consensus.cc:388] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:32.279196 13518 raft_consensus.cc:482] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:32.279315 13518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:32.280642 13518 raft_consensus.cc:504] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } }
I20260929 18:40:32.281101 13518 leader_election.cc:302] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a37efa6e3c5549dfa532fc2feb30dcc4; no voters:
I20260929 18:40:32.281523 13518 leader_election.cc:288] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:32.281639 13523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:32.281905 13523 raft_consensus.cc:686] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [term 1 LEADER]: Becoming Leader. State: Replica: a37efa6e3c5549dfa532fc2feb30dcc4, State: Running, Role: LEADER
I20260929 18:40:32.282310 13523 consensus_queue.cc:237] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } }
I20260929 18:40:32.282888 13518 sys_catalog.cc:564] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:32.284304 13524 sys_catalog.cc:455] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } } }
I20260929 18:40:32.284446 13524 sys_catalog.cc:458] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:32.284915 13525 sys_catalog.cc:455] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a37efa6e3c5549dfa532fc2feb30dcc4. Latest consensus state: current_term: 1 leader_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37efa6e3c5549dfa532fc2feb30dcc4" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39097 } } }
I20260929 18:40:32.284973 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.269921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.11.118.254:39097)
0929 18:40:32.270382 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:32.270420 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:32.270471 (+ 51us) server_negotiation.cc:407] Connection header received
0929 18:40:32.271143 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:32.271161 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:32.271621 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:32.272022 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:32.272923 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.273993 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.275279 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.275778 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.275906 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.278927 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:32.278933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:32.278952 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:32.278992 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:32.281215 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.281836 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.281846 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.281855 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.282089 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.282512 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.282514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.282516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.283035 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:32.283240 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:32.283602 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 18:40:32.283921 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":164,"threads_started":1}
I20260929 18:40:32.285085 13525 sys_catalog.cc:458] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4 [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:32.288110 13538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:32.288215 13538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:32.288283 13532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:32.288972 13532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:32.295068 13532 catalog_manager.cc:1347] Generated new cluster ID: 402a2b1d149f4d01b285cfff9ac7f6a9
I20260929 18:40:32.295161 13532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:32.324057 13532 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:32.325088 13532 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:32.334779 13532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Generated new TSK 0
I20260929 18:40:32.335608 13532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:32.370501 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39097
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:32.485299 13544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:32.485622 13544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:32.485730 13544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:32.489956 13544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:32.490065 13544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:32.490144 13544 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 18:40:32.490187 13544 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:32.490257 13544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:32.495743 13544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:32.496963 13544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:32.498018 13544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:32.508244 13544 server_base.cc:1034] running on GCE node
W20260929 18:40:32.508460 13550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.508443 13552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.508502 13549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.508989 13544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:32.509753 13544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:32.510942 13544 hybrid_clock.cc:648] HybridClock initialized: now 1790707232510930 us; error 27 us; skew 500 ppm
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:32.513900 13544 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:32.514938 13544 webserver.cc:460] Webserver started at http://127.11.118.193:39773/ using document root <none> and password file <none>
I20260929 18:40:32.515617 13544 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:32.515726 13544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:32.515969 13544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:32.518147 13544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "c70183fe74b942158c75ca5cb9a1c94d"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d67188d1f6b13b51b6ab191c0f8012a8"
server_key_iv: "cb36307d78a242ee9baed1f6b6493ecc"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.518807 13544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "c70183fe74b942158c75ca5cb9a1c94d"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d67188d1f6b13b51b6ab191c0f8012a8"
server_key_iv: "cb36307d78a242ee9baed1f6b6493ecc"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.522904 13544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:32.525490 13559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.527071 13544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:32.527221 13544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "c70183fe74b942158c75ca5cb9a1c94d"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d67188d1f6b13b51b6ab191c0f8012a8"
server_key_iv: "cb36307d78a242ee9baed1f6b6493ecc"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.527357 13544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:32.565727 13544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:32.569077 13544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:32.569312 13544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:32.570935 13544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:32.571019 13544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.571086 13544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:32.571138 13544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:32.582216 13544 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:33281
I20260929 18:40:32.582324 13674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:33281 every 8 connection(s)
I20260929 18:40:32.583438 13544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:32.585536 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.571794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:53553 (local address 127.11.118.254:39097)
0929 18:40:32.571930 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:32.571933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:32.573004 (+ 1071us) server_negotiation.cc:407] Connection header received
0929 18:40:32.574107 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:32.574113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:32.574210 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:32.574276 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:32.575789 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.576706 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.578096 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.578581 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.578704 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.581508 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:32.581512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:32.581517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:32.581552 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:32.583309 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.583902 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.583912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.583914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.584059 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.584492 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.584496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.584497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.584699 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:32.584776 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:32.585295 (+ 519us) server_negotiation.cc:299] Negotiation successful
0929 18:40:32.585417 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 18:40:32.586410 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.572327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.193:53553)
0929 18:40:32.572830 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:32.572872 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:32.573792 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:32.574681 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:32.574687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:32.575042 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:32.575611 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.575629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.576842 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.576847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.577954 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.577963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.578758 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.578764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.579047 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.579687 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:32.579703 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:32.581330 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:32.583436 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.583440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.583451 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.583755 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.584200 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.584203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.584205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.584380 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.584802 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:32.584807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:32.585096 (+ 289us) client_negotiation.cc:769] Sending connection context
0929 18:40:32.585246 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 18:40:32.585486 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260929 18:40:32.588080 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13544
I20260929 18:40:32.588071 13675 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39097
I20260929 18:40:32.588179 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:32.588337 13675 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:32.588497 11739 external_mini_cluster.cc:1428] Setting key fc5ba2fbdc9b117b9c81333625aa3882
I20260929 18:40:32.588886 13675 heartbeater.cc:511] Master 127.11.118.254:39097 requested a full tablet report, sending...
I20260929 18:40:32.590627 13471 ts_manager.cc:194] Registered new tserver with Master: c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281)
I20260929 18:40:32.592070 13471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:53553
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:32.599370 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.591410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:50853 (local address 127.11.118.254:39097)
0929 18:40:32.591518 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:32.591523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:32.591832 (+ 309us) server_negotiation.cc:407] Connection header received
0929 18:40:32.592007 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:32.592010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:32.592058 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:32.592150 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:32.592604 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.593204 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.594386 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.594692 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.594769 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.596085 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:32.596089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:32.596092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:32.596118 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:32.597600 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.598118 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.598122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.598123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.598205 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.598534 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.598537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.598539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.598724 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:32.598831 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:32.599089 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 18:40:32.599199 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 18:40:32.599390 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.591346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.193:50853)
0929 18:40:32.591740 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:32.591758 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:32.591915 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:32.592168 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:32.592171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:32.592385 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:32.592487 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.592492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.593361 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.593365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.594209 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.594219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.594924 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.594926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.595003 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.595575 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:32.595587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:32.595925 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:32.597733 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.597737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.597740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.598019 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.598337 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.598340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.598342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.598432 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.598819 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:32.598822 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:32.598923 (+ 101us) client_negotiation.cc:769] Sending connection context
0929 18:40:32.599037 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:40:32.599191 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:32.618530 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39097
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:32.730355 13684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:32.730621 13684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:32.730669 13684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:32.735025 13684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:32.735085 13684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:32.735126 13684 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 18:40:32.735164 13684 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:32.735225 13684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:32.740569 13684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:32.741683 13684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:32.742745 13684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:32.752691 13684 server_base.cc:1034] running on GCE node
W20260929 18:40:32.752651 13690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.752651 13692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.753048 13689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.753473 13684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:32.754072 13684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:32.755241 13684 hybrid_clock.cc:648] HybridClock initialized: now 1790707232755228 us; error 69 us; skew 500 ppm
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:32.758157 13684 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:32.759172 13684 webserver.cc:460] Webserver started at http://127.11.118.194:37365/ using document root <none> and password file <none>
I20260929 18:40:32.759768 13684 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:32.759858 13684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:32.760092 13684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:32.762094 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "2489e38864c04dbe856ad181f54a64c8"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d3513fa14d75b754c662b952c8135dfa"
server_key_iv: "d1dc734c83b79686ce1e18904390d14c"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.762717 13684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "2489e38864c04dbe856ad181f54a64c8"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d3513fa14d75b754c662b952c8135dfa"
server_key_iv: "d1dc734c83b79686ce1e18904390d14c"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.766325 13684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 18:40:32.768647 13699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.769855 13684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 18:40:32.769994 13684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "2489e38864c04dbe856ad181f54a64c8"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d3513fa14d75b754c662b952c8135dfa"
server_key_iv: "d1dc734c83b79686ce1e18904390d14c"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.770112 13684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:32.785351 13684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:32.788803 13684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:32.789023 13684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:32.790539 13684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:32.790606 13684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.790678 13684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:32.790727 13684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:32.801396 13684 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:37621
I20260929 18:40:32.801422 13814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:37621 every 8 connection(s)
I20260929 18:40:32.802332 13684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:32.804908 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13684
I20260929 18:40:32.805018 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:32.805055 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.791476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:35903 (local address 127.11.118.254:39097)
0929 18:40:32.791632 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:32.791636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:32.792495 (+ 859us) server_negotiation.cc:407] Connection header received
0929 18:40:32.793643 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:32.793650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:32.793716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:32.793825 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:32.795208 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.796101 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.797444 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.797846 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.797973 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.800931 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:32.800935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:32.800937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:32.800979 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:32.802690 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.803337 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.803340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.803342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.803420 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.803831 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.803834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.803835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.804078 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:32.804168 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:32.804685 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 18:40:32.804822 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 18:40:32.805341 11739 external_mini_cluster.cc:1428] Setting key f97b158b675f9d7eec489378e23977d0
I20260929 18:40:32.805904 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.791897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.194:35903)
0929 18:40:32.792331 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:32.792380 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:32.793391 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:32.794061 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:32.794070 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:32.794553 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:32.795051 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.795064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.796253 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.796256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.797341 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.797350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.798058 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.798063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.798252 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.798937 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:32.798953 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:32.800729 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:32.802811 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.802817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.802832 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.803221 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.803562 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.803564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.803566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.803710 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.804192 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:32.804197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:32.804433 (+ 236us) client_negotiation.cc:769] Sending connection context
0929 18:40:32.804593 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 18:40:32.804862 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":117,"threads_started":1}
I20260929 18:40:32.807111 13815 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39097
I20260929 18:40:32.807277 13815 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:32.807767 13815 heartbeater.cc:511] Master 127.11.118.254:39097 requested a full tablet report, sending...
I20260929 18:40:32.808911 13472 ts_manager.cc:194] Registered new tserver with Master: 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621)
I20260929 18:40:32.809651 13472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:35903
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:32.817652 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.809659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:41065 (local address 127.11.118.254:39097)
0929 18:40:32.809785 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:32.809789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:32.810052 (+ 263us) server_negotiation.cc:407] Connection header received
0929 18:40:32.810186 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:32.810191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:32.810237 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:32.810362 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:32.810837 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.811332 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.812556 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.812838 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.812946 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.814350 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:32.814353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:32.814355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:32.814383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:32.815900 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.816342 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.816347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.816349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.816431 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:32.816714 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:32.816717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:32.816719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:32.816996 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:32.817110 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:32.817361 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 18:40:32.817467 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:40:32.817723 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:32.809592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.194:41065)
0929 18:40:32.809980 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:32.809996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:32.810096 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:32.810380 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:32.810383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:32.810615 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:32.810721 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.810728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.811457 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.811460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.812414 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:32.812428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:32.812982 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:32.812988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:32.813099 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:32.813623 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:32.813643 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:32.814156 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:32.816033 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.816036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.816038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.816240 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.816540 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:32.816542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:32.816544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:32.816617 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:32.817097 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:32.817100 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:32.817196 (+ 96us) client_negotiation.cc:769] Sending connection context
0929 18:40:32.817300 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 18:40:32.817456 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:32.837381 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39097
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:32.952176 13824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:32.952497 13824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:32.952548 13824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:32.956905 13824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:32.956996 13824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:32.957039 13824 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 18:40:32.957054 13824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:40:32.957118 13824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:32.962872 13824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:32.964052 13824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:32.965147 13824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:32.975688 13832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:32.975690 13829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.975749 13824 server_base.cc:1034] running on GCE node
W20260929 18:40:32.975690 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:32.976423 13824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:32.977182 13824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:32.978379 13824 hybrid_clock.cc:648] HybridClock initialized: now 1790707232978357 us; error 36 us; skew 500 ppm
Sep 29 18:40:32 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:32.981531 13824 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:32.982681 13824 webserver.cc:460] Webserver started at http://127.11.118.195:35773/ using document root <none> and password file <none>
I20260929 18:40:32.983363 13824 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:32.983457 13824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:32.983687 13824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:32.986053 13824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "2cd58d002bb049c59766048af325ab22"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d9d24fa074ef34f19ff9c1fc179267b0"
server_key_iv: "3635dcd60d17589dc33a3f74f294c606"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.986711 13824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd58d002bb049c59766048af325ab22"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d9d24fa074ef34f19ff9c1fc179267b0"
server_key_iv: "3635dcd60d17589dc33a3f74f294c606"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.990856 13824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:32.993546 13839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:32.994892 13824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:32.995041 13824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2cd58d002bb049c59766048af325ab22"
format_stamp: "Formatted at 2026-09-29 18:40:32 on dist-test-slave-cgv7"
server_key: "d9d24fa074ef34f19ff9c1fc179267b0"
server_key_iv: "3635dcd60d17589dc33a3f74f294c606"
server_key_version: "encryptionkey@0"
I20260929 18:40:32.995167 13824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:33.013507 13824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:33.017176 13824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:33.017335 13824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:33.018846 13824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:33.018899 13824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:33.018942 13824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:33.018996 13824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:40:33 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:33.030069 13824 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:39797
I20260929 18:40:33.030205 13954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:39797 every 8 connection(s)
I20260929 18:40:33.031148 13824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:33.033591 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.019670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:53065 (local address 127.11.118.254:39097)
0929 18:40:33.019852 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.019858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.020776 (+ 918us) server_negotiation.cc:407] Connection header received
0929 18:40:33.022148 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.022154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.022222 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.022312 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:33.024335 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.025007 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.026448 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.026770 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.026897 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.029497 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:33.029501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:33.029504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:33.029539 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:33.031328 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.031956 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.031961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.031963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.032061 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.032475 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.032480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.032481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.032677 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:33.032791 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.033281 (+ 490us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.033417 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:40:33.034161 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 13824
I20260929 18:40:33.034319 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:33.034637 11739 external_mini_cluster.cc:1428] Setting key f3f8658a5ec51edbb5d3ebd63db84d9a
I20260929 18:40:33.034613 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.020094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.195:53065)
0929 18:40:33.020580 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.020642 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.021852 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.022809 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.022817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.023337 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:33.024113 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.024136 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.025135 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.025138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.026318 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.026328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.026950 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.026954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.027144 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.027762 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:33.027778 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:33.029321 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:33.031442 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.031447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.031462 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.031863 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.032179 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.032182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.032184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.032376 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.032786 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:33.032792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:33.033044 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.033233 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.033552 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":99,"threads_started":1}
I20260929 18:40:33.035982 13955 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39097
I20260929 18:40:33.036127 13955 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:33.036684 13955 heartbeater.cc:511] Master 127.11.118.254:39097 requested a full tablet report, sending...
I20260929 18:40:33.037613 13472 ts_manager.cc:194] Registered new tserver with Master: 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:33.038272 13472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:53065
I20260929 18:40:33.045274 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.038208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:33065 (local address 127.11.118.254:39097)
0929 18:40:33.038372 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.038376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.038589 (+ 213us) server_negotiation.cc:407] Connection header received
0929 18:40:33.038703 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.038706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.038749 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.038839 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:33.039268 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.039918 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.040752 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.041008 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.041119 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.042106 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:33.042110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:33.042112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:33.042136 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:33.043652 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.044086 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.044090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.044092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.044176 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.044457 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.044462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.044463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.044643 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:33.044750 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.045005 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.045113 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 18:40:33.045287 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.038106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.11.118.195:33065)
0929 18:40:33.038505 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.038521 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.038612 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.038854 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.038857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.039052 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:33.039154 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.039160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.040040 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.040043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.040633 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.040642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.041130 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.041133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.041237 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.041660 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:33.041671 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:33.041973 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:33.043782 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.043785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.043787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.043985 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.044274 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.044277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.044279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.044366 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.044739 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:33.044742 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:33.044839 (+ 97us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.044943 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.045086 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":87,"threads_started":1}
I20260929 18:40:33.048585 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 18:40:33 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 18:40:33 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:33.075063 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.066025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.11.118.254:39097)
0929 18:40:33.066175 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.066179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.066192 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:40:33.066237 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.066239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.066339 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.066423 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:33.067123 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.067649 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.068468 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.068692 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.068783 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.071564 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:33.071568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:33.071570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:33.071601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:33.073270 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.073716 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.073720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.073722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.073804 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.074097 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.074102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.074105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.074374 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:33.074518 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.074763 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.074883 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:40:33.078264 13472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:33.081231 13472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:33.095573 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.089941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.11.118.195:39797)
0929 18:40:33.091100 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.091112 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.091133 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:40:33.091195 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.091198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.091358 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.091527 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:33.091814 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.092711 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.094180 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.094835 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.095022 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.095162 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.095324 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.095417 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":114,"threads_started":1}
I20260929 18:40:33.095647 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.089750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:39797 (local address 127.0.0.1:32946)
0929 18:40:33.090167 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.090221 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.090401 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.091536 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.091541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.091562 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:33.091700 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.091706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.092815 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.092821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.094063 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.094072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.095017 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.095020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.095127 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.095168 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.095282 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.095396 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":51,"thread_start_us":77,"threads_started":1}
I20260929 18:40:33.096213 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.090489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:37621 (local address 127.0.0.1:56998)
0929 18:40:33.090743 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.090758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.090853 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.092172 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.092175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.092194 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:33.092274 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.092279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.093545 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.093548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.094732 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.094741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.095834 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.095838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.095920 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.095970 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.096022 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.096092 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":4,"thread_start_us":127,"threads_started":1}
I20260929 18:40:33.096357 13974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.090179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.0.0.1:48920)
0929 18:40:33.090702 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.090714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.090792 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.091673 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.091676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.091694 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:33.091801 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.091807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.093359 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.093362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.094609 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.094619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.095863 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.095867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.096002 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.096024 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.096091 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.096173 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":24,"thread_start_us":118,"threads_started":1}
I20260929 18:40:33.096913 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.091335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.11.118.194:37621)
0929 18:40:33.091728 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.091739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.091758 (+ 19us) server_negotiation.cc:407] Connection header received
0929 18:40:33.091828 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.091836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.092024 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.092175 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:33.092368 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.093431 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.094854 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.095688 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.096445 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.096615 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.096661 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.096740 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":104,"threads_started":1}
I20260929 18:40:33.097250 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.090950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.11.118.193:33281)
0929 18:40:33.091193 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.091201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.091225 (+ 24us) server_negotiation.cc:407] Connection header received
0929 18:40:33.091300 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.091306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.091516 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.091945 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:33.092080 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.093229 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.094766 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.095718 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.096874 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.096996 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.097043 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.097122 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260929 18:40:33.099001 13889 tablet_service.cc:1467] Processing CreateTablet for tablet 6010bc89f3514831a30dbd21872dd206 (DEFAULT_TABLE table=test-table [id=9d056f203eb4471fa9590edfbf2601ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.100144 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6010bc89f3514831a30dbd21872dd206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.099772 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 6010bc89f3514831a30dbd21872dd206 (DEFAULT_TABLE table=test-table [id=9d056f203eb4471fa9590edfbf2601ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.100735 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6010bc89f3514831a30dbd21872dd206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.103794 13749 tablet_service.cc:1467] Processing CreateTablet for tablet 6010bc89f3514831a30dbd21872dd206 (DEFAULT_TABLE table=test-table [id=9d056f203eb4471fa9590edfbf2601ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.104610 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6010bc89f3514831a30dbd21872dd206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.107918 13979 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Bootstrap starting.
I20260929 18:40:33.110394 13979 tablet_bootstrap.cc:654] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.110679 13978 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Bootstrap starting.
I20260929 18:40:33.111317 13979 log.cc:824] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:33.112433 13981 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Bootstrap starting.
I20260929 18:40:33.112818 13978 tablet_bootstrap.cc:654] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.113476 13979 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: No bootstrap required, opened a new log
I20260929 18:40:33.113651 13978 log.cc:824] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:33.113699 13979 ts_tablet_manager.cc:1397] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260929 18:40:33.114457 13981 tablet_bootstrap.cc:654] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.115222 13981 log.cc:824] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:33.116271 13978 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: No bootstrap required, opened a new log
I20260929 18:40:33.116463 13978 ts_tablet_manager.cc:1397] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:40:33.117837 13981 tablet_bootstrap.cc:492] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: No bootstrap required, opened a new log
I20260929 18:40:33.117694 13979 raft_consensus.cc:348] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.117974 13979 raft_consensus.cc:374] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.118026 13979 raft_consensus.cc:729] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.118033 13981 ts_tablet_manager.cc:1397] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260929 18:40:33.118535 13979 consensus_queue.cc:260] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.119253 13955 heartbeater.cc:503] Master 127.11.118.254:39097 was elected leader, sending a full tablet report...
I20260929 18:40:33.119592 13979 ts_tablet_manager.cc:1428] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:40:33.120779 13978 raft_consensus.cc:348] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.121047 13978 raft_consensus.cc:374] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.121101 13978 raft_consensus.cc:729] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.121589 13978 consensus_queue.cc:260] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.122329 13675 heartbeater.cc:503] Master 127.11.118.254:39097 was elected leader, sending a full tablet report...
I20260929 18:40:33.122690 13978 ts_tablet_manager.cc:1428] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:40:33.123862 13981 raft_consensus.cc:348] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.124090 13981 raft_consensus.cc:374] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.124158 13981 raft_consensus.cc:729] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2489e38864c04dbe856ad181f54a64c8, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.124670 13981 consensus_queue.cc:260] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.125384 13815 heartbeater.cc:503] Master 127.11.118.254:39097 was elected leader, sending a full tablet report...
I20260929 18:40:33.125679 13981 ts_tablet_manager.cc:1428] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260929 18:40:33.267254 13985 raft_consensus.cc:482] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:33.267510 13985 raft_consensus.cc:504] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.268720 13985 leader_election.cc:288] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797), 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621)
I20260929 18:40:33.273973 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.269567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:60129 (local address 127.11.118.194:37621)
0929 18:40:33.269707 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.269713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.269903 (+ 190us) server_negotiation.cc:407] Connection header received
0929 18:40:33.270034 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.270038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.270112 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.270189 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:33.270524 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.271177 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.272528 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.273163 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.273266 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.273315 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.273785 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.273852 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:40:33.273962 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.269086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:48079 (local address 127.11.118.195:39797)
0929 18:40:33.269245 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.269250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.269410 (+ 160us) server_negotiation.cc:407] Connection header received
0929 18:40:33.269610 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.269613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.269671 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.269715 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:33.270113 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.270915 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.272493 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.273126 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.273265 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.273315 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.273757 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.273837 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 18:40:33.273993 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.269471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:37621 (local address 127.11.118.193:60129)
0929 18:40:33.269808 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.269818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.269943 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.270217 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.270220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.270237 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:33.270404 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.270411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.271291 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.271294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.272408 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.272419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.273356 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.273364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.273521 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.273618 (+ 97us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.273713 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.273792 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":87,"thread_start_us":83,"threads_started":1}
I20260929 18:40:33.273993 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.268955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:39797 (local address 127.11.118.193:48079)
0929 18:40:33.269311 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.269327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.269492 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.269852 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.269855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.269870 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:33.269983 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.269988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.271022 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.271024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.272330 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.272342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.273356 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.273364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.273555 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.273618 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.273709 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.273808 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":54,"thread_start_us":173,"threads_started":1}
I20260929 18:40:33.274868 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6010bc89f3514831a30dbd21872dd206" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22" is_pre_election: true
I20260929 18:40:33.274868 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6010bc89f3514831a30dbd21872dd206" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8" is_pre_election: true
I20260929 18:40:33.275207 13909 raft_consensus.cc:2454] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.275207 13769 raft_consensus.cc:2454] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.275688 13560 leader_election.cc:302] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.275950 13985 raft_consensus.cc:2781] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:33.276036 13985 raft_consensus.cc:482] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:33.276081 13985 raft_consensus.cc:3037] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.277159 13985 raft_consensus.cc:504] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
I20260929 18:40:33.277529 13985 leader_election.cc:288] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Requested vote from peers 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797), 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621)
I20260929 18:40:33.277902 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6010bc89f3514831a30dbd21872dd206" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22"
I20260929 18:40:33.278025 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6010bc89f3514831a30dbd21872dd206" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8"
I20260929 18:40:33.278067 13909 raft_consensus.cc:3037] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.278165 13769 raft_consensus.cc:3037] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.279256 13769 raft_consensus.cc:2454] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.279289 13909 raft_consensus.cc:2454] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.279577 13560 leader_election.cc:302] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.279819 13985 raft_consensus.cc:2781] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:33.280109 13985 raft_consensus.cc:686] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Becoming Leader. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Running, Role: LEADER
I20260929 18:40:33.280458 13985 consensus_queue.cc:237] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } }
W20260929 18:40:33.282848 13956 tablet.cc:2369] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:33.283891 13472 catalog_manager.cc:5495] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d reported cstate change: term changed from 0 to 1, leader changed from <none> to c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193). New cstate: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } health_report { overall_health: HEALTHY } } }
W20260929 18:40:33.303738 13816 tablet.cc:2369] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:40:33.335110 13676 tablet.cc:2369] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 18:40:33 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:33.357219 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.348199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39097 (local address 127.0.0.1:37920)
0929 18:40:33.348541 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.348554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.348710 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.349045 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.349050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.349479 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:33.349577 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.349583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.350396 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.350403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.350965 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.350972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.351409 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.351412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.351482 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.352062 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:33.352074 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:33.353603 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:33.355533 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.355543 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.355549 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.355804 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.356109 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:33.356112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:33.356114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:33.356202 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:33.356648 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:33.356653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:33.356767 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.356866 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.357015 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":124,"threads_started":1}
I20260929 18:40:33.357218 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.348209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.11.118.254:39097)
0929 18:40:33.348396 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.348401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.348648 (+ 247us) server_negotiation.cc:407] Connection header received
0929 18:40:33.348820 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.348824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.348868 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.348958 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:33.349709 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.350241 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.351088 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.351299 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.351404 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.353777 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:33.353781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:33.353783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:33.353805 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:33.355397 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.355905 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.355908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.355909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.355986 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:33.356316 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:33.356319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:33.356320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:33.356540 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:33.356627 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.356929 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.357083 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":79}
I20260929 18:40:33.361600 13472 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:37920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 18:40:33.362512 13472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:33.367774 13749 tablet_service.cc:1467] Processing CreateTablet for tablet bdf74701b0b643cf8487aeb4be56d57d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa51981ce2ea4c1a900a68e976d1df90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:33.367774 13609 tablet_service.cc:1467] Processing CreateTablet for tablet bdf74701b0b643cf8487aeb4be56d57d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa51981ce2ea4c1a900a68e976d1df90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:33.367918 13889 tablet_service.cc:1467] Processing CreateTablet for tablet bdf74701b0b643cf8487aeb4be56d57d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa51981ce2ea4c1a900a68e976d1df90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:40:33.368477 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdf74701b0b643cf8487aeb4be56d57d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.368476 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdf74701b0b643cf8487aeb4be56d57d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.368499 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdf74701b0b643cf8487aeb4be56d57d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.371057 13981 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: Bootstrap starting.
I20260929 18:40:33.371331 13979 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: Bootstrap starting.
I20260929 18:40:33.372061 13981 tablet_bootstrap.cc:654] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.372283 13979 tablet_bootstrap.cc:654] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.372929 13978 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: Bootstrap starting.
I20260929 18:40:33.373514 13979 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: No bootstrap required, opened a new log
I20260929 18:40:33.373523 13981 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: No bootstrap required, opened a new log
I20260929 18:40:33.373596 13979 ts_tablet_manager.cc:1397] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:33.373596 13981 ts_tablet_manager.cc:1397] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260929 18:40:33.374231 13978 tablet_bootstrap.cc:654] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.374166 13979 raft_consensus.cc:348] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.374310 13979 raft_consensus.cc:374] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.374347 13979 raft_consensus.cc:729] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.374507 13979 consensus_queue.cc:260] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.374382 13981 raft_consensus.cc:348] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.374624 13981 raft_consensus.cc:374] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.374665 13981 raft_consensus.cc:729] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2489e38864c04dbe856ad181f54a64c8, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.374842 13981 consensus_queue.cc:260] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.374850 13984 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.374981 13984 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.375134 13986 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.375257 13986 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.375641 13979 ts_tablet_manager.cc:1428] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 18:40:33.376039 13978 tablet_bootstrap.cc:492] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: No bootstrap required, opened a new log
I20260929 18:40:33.376127 13978 ts_tablet_manager.cc:1397] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:33.376598 13978 raft_consensus.cc:348] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.376714 13978 raft_consensus.cc:374] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.376745 13978 raft_consensus.cc:729] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.376850 13978 consensus_queue.cc:260] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.377121 13978 ts_tablet_manager.cc:1428] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:33.377111 13985 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.377234 13985 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.377642 13981 ts_tablet_manager.cc:1428] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 18:40:33.502820 13985 raft_consensus.cc:482] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:33.502990 13985 raft_consensus.cc:504] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.503533 13985 leader_election.cc:288] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:33.504094 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdf74701b0b643cf8487aeb4be56d57d" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8" is_pre_election: true
I20260929 18:40:33.504094 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdf74701b0b643cf8487aeb4be56d57d" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22" is_pre_election: true
I20260929 18:40:33.504257 13769 raft_consensus.cc:2454] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.504257 13909 raft_consensus.cc:2454] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.504570 13560 leader_election.cc:302] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.504741 13985 raft_consensus.cc:2781] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:33.504801 13985 raft_consensus.cc:482] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:33.504839 13985 raft_consensus.cc:3037] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.505983 13985 raft_consensus.cc:504] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.506373 13985 leader_election.cc:288] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Requested vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:33.507830 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdf74701b0b643cf8487aeb4be56d57d" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8"
I20260929 18:40:33.507830 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdf74701b0b643cf8487aeb4be56d57d" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22"
I20260929 18:40:33.507975 13909 raft_consensus.cc:3037] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.507975 13769 raft_consensus.cc:3037] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.509133 13909 raft_consensus.cc:2454] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.509133 13769 raft_consensus.cc:2454] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.509462 13560 leader_election.cc:302] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.509718 13985 raft_consensus.cc:2781] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:33.509812 13985 raft_consensus.cc:686] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Becoming Leader. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Running, Role: LEADER
I20260929 18:40:33.509977 13985 consensus_queue.cc:237] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.510762 13989 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: TxnStatusTablet state changed. Reason: New leader c70183fe74b942158c75ca5cb9a1c94d. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.510877 13989 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:33.511196 14001 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 18:40:33.511468 14001 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 18:40:33.511873 13470 catalog_manager.cc:5495] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d reported cstate change: term changed from 0 to 1, leader changed from <none> to c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193). New cstate: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:33.542724 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.537691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.0.0.1:48922)
0929 18:40:33.537889 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.537908 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.538026 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.538753 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.538756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.538765 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:33.538870 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.538875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.539918 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.539923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.540704 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.540716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.541360 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.541364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.541448 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.541504 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:33.542388 (+ 884us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:33.542415 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.542483 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.542598 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 18:40:33.542725 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.538228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.11.118.193:33281)
0929 18:40:33.538403 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.538409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.538427 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:40:33.538508 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.538511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.538603 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.538714 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:33.538992 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.539797 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.540839 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.541214 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.541317 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.541678 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:33.542247 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:33.542395 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.542509 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.542581 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 18:40:33.548295 13769 raft_consensus.cc:1261] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.548261 13909 raft_consensus.cc:1261] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.549005 13985 consensus_queue.cc:1035] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:33.549280 13989 consensus_queue.cc:1035] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:33.552922 13986 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: TxnStatusTablet state changed. Reason: New leader c70183fe74b942158c75ca5cb9a1c94d. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.553064 13986 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.553308 13984 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: TxnStatusTablet state changed. Reason: New leader c70183fe74b942158c75ca5cb9a1c94d. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.553439 13984 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.557346 13985 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.557495 13985 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:33.557631 13985 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.557714 13985 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P c70183fe74b942158c75ca5cb9a1c94d: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:40:33.557880 13986 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.557991 13986 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2489e38864c04dbe856ad181f54a64c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.559216 13984 tablet_replica.cc:439] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } } }
I20260929 18:40:33.559365 13984 tablet_replica.cc:442] T bdf74701b0b643cf8487aeb4be56d57d P 2cd58d002bb049c59766048af325ab22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:40:33.561026 14005 mvcc.cc:205] Tried to move back new op lower bound from 7334736828606558208 to 7334736828458569728. Current Snapshot: MvccSnapshot[applied={T|T < 7334736828606558208}]
I20260929 18:40:33.562685 14006 mvcc.cc:205] Tried to move back new op lower bound from 7334736828606558208 to 7334736828458569728. Current Snapshot: MvccSnapshot[applied={T|T < 7334736828606558208}]
I20260929 18:40:33.562898 14004 mvcc.cc:205] Tried to move back new op lower bound from 7334736828606558208 to 7334736828458569728. Current Snapshot: MvccSnapshot[applied={T|T < 7334736828606558208 or (T in {7334736828606558208})}]
I20260929 18:40:33.568150 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.564018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.11.118.193:33281)
0929 18:40:33.564126 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.564130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.564144 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:40:33.564189 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.564192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.564259 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.564336 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:33.564735 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.565670 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.566998 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.567321 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.567423 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.567741 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:33.567832 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:33.567951 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.567987 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.568038 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 18:40:33.580667 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.576918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:37717 (local address 127.11.118.193:33281)
0929 18:40:33.577062 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:33.577067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:33.577302 (+ 235us) server_negotiation.cc:407] Connection header received
0929 18:40:33.577431 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:33.577434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:33.577486 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:33.577561 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:33.577800 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.578605 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.579483 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.579812 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.579913 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.580226 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:33.580311 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:33.580381 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:33.580455 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 18:40:33.580506 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:40:33.580672 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:33.576815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.11.118.193:37717)
0929 18:40:33.577203 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:33.577227 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:33.577326 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:33.577577 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:33.577580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:33.577587 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:33.577695 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.577700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.578723 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.578726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.579374 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:33.579382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:33.579939 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:33.579942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:33.580026 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:33.580074 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:33.580391 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:33.580396 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:40:33.580445 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 18:40:33.580528 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":167,"threads_started":1}
I20260929 18:40:33.588654 13769 raft_consensus.cc:1261] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.589056 13985 consensus_queue.cc:1035] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:33.589655 13909 raft_consensus.cc:1261] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.590035 13985 consensus_queue.cc:1035] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:33.634996 13470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:33.635288 13470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:33.637907 13470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Sending DeleteTablet for 3 replicas of tablet 6010bc89f3514831a30dbd21872dd206
I20260929 18:40:33.638778 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet 6010bc89f3514831a30dbd21872dd206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:33 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32946
I20260929 18:40:33.639292 14020 tablet_replica.cc:330] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: stopping tablet replica
I20260929 18:40:33.639310 13749 tablet_service.cc:1514] Processing DeleteTablet for tablet 6010bc89f3514831a30dbd21872dd206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:33 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:56998
I20260929 18:40:33.639544 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 6010bc89f3514831a30dbd21872dd206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:33 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48920
I20260929 18:40:33.639721 14021 tablet_replica.cc:330] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: stopping tablet replica
I20260929 18:40:33.639915 14020 raft_consensus.cc:2229] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:33.640012 14021 raft_consensus.cc:2229] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:33.640219 14021 raft_consensus.cc:2258] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:33.640219 14020 raft_consensus.cc:2258] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:33.640174 13470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:33.640604 13470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:33.641741 14022 tablet_replica.cc:330] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: stopping tablet replica
I20260929 18:40:33.641872 14020 ts_tablet_manager.cc:1905] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:33.642050 14022 raft_consensus.cc:2229] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:33.642436 14022 raft_consensus.cc:2258] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:33.644685 14022 ts_tablet_manager.cc:1905] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:33.645637 14021 ts_tablet_manager.cc:1905] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:33.645772 13609 tablet_service.cc:1467] Processing CreateTablet for tablet cbb094505edf4ce0ae9d781cab0429df (DEFAULT_TABLE table=test-table [id=c7f1bad1db594de58f9a01516d0d3063]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.645792 13749 tablet_service.cc:1467] Processing CreateTablet for tablet cbb094505edf4ce0ae9d781cab0429df (DEFAULT_TABLE table=test-table [id=c7f1bad1db594de58f9a01516d0d3063]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.646052 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb094505edf4ce0ae9d781cab0429df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.646425 13889 tablet_service.cc:1467] Processing CreateTablet for tablet cbb094505edf4ce0ae9d781cab0429df (DEFAULT_TABLE table=test-table [id=c7f1bad1db594de58f9a01516d0d3063]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:33.646668 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb094505edf4ce0ae9d781cab0429df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.648448 13981 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Bootstrap starting.
I20260929 18:40:33.649119 13979 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Bootstrap starting.
I20260929 18:40:33.649384 13981 tablet_bootstrap.cc:654] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.649528 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb094505edf4ce0ae9d781cab0429df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:33.650076 13979 tablet_bootstrap.cc:654] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.651084 14020 ts_tablet_manager.cc:1918] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:33.651176 14020 log.cc:1196] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6010bc89f3514831a30dbd21872dd206
I20260929 18:40:33.651521 14020 ts_tablet_manager.cc:1939] T 6010bc89f3514831a30dbd21872dd206 P 2cd58d002bb049c59766048af325ab22: Deleting consensus metadata
I20260929 18:40:33.652130 13978 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Bootstrap starting.
I20260929 18:40:33.652737 13456 catalog_manager.cc:4841] TS 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797): tablet 6010bc89f3514831a30dbd21872dd206 (table test-table [id=9d056f203eb4471fa9590edfbf2601ca]) successfully deleted
I20260929 18:40:33.653118 13978 tablet_bootstrap.cc:654] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:33.653656 14021 ts_tablet_manager.cc:1918] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:33.653751 14021 log.cc:1196] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/6010bc89f3514831a30dbd21872dd206
I20260929 18:40:33.654073 14021 ts_tablet_manager.cc:1939] T 6010bc89f3514831a30dbd21872dd206 P 2489e38864c04dbe856ad181f54a64c8: Deleting consensus metadata
I20260929 18:40:33.655380 13456 catalog_manager.cc:4841] TS 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621): tablet 6010bc89f3514831a30dbd21872dd206 (table test-table [id=9d056f203eb4471fa9590edfbf2601ca]) successfully deleted
I20260929 18:40:33.655470 13978 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: No bootstrap required, opened a new log
I20260929 18:40:33.655584 13978 ts_tablet_manager.cc:1397] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:33.656078 13978 raft_consensus.cc:348] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.656210 13978 raft_consensus.cc:374] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.656251 13978 raft_consensus.cc:729] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.656386 13979 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: No bootstrap required, opened a new log
I20260929 18:40:33.656373 13978 consensus_queue.cc:260] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.656507 13979 ts_tablet_manager.cc:1397] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 18:40:33.656647 13978 ts_tablet_manager.cc:1428] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:33.657032 13979 raft_consensus.cc:348] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.657147 13979 raft_consensus.cc:374] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.657191 13979 raft_consensus.cc:729] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.657325 13979 consensus_queue.cc:260] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.655469 14022 ts_tablet_manager.cc:1918] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:33.657493 14022 log.cc:1196] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/6010bc89f3514831a30dbd21872dd206
I20260929 18:40:33.657570 13979 ts_tablet_manager.cc:1428] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260929 18:40:33.657859 14022 ts_tablet_manager.cc:1939] T 6010bc89f3514831a30dbd21872dd206 P c70183fe74b942158c75ca5cb9a1c94d: Deleting consensus metadata
I20260929 18:40:33.658967 13457 catalog_manager.cc:4841] TS c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281): tablet 6010bc89f3514831a30dbd21872dd206 (table test-table [id=9d056f203eb4471fa9590edfbf2601ca]) successfully deleted
I20260929 18:40:33.659653 13981 tablet_bootstrap.cc:492] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: No bootstrap required, opened a new log
I20260929 18:40:33.659731 13981 ts_tablet_manager.cc:1397] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260929 18:40:33.660127 13981 raft_consensus.cc:348] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.660221 13981 raft_consensus.cc:374] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:33.660243 13981 raft_consensus.cc:729] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2489e38864c04dbe856ad181f54a64c8, State: Initialized, Role: FOLLOWER
I20260929 18:40:33.660379 13981 consensus_queue.cc:260] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.660645 13981 ts_tablet_manager.cc:1428] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:40:33.751072 14018 raft_consensus.cc:482] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:33.751240 14018 raft_consensus.cc:504] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.751711 14018 leader_election.cc:288] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:33.752311 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb094505edf4ce0ae9d781cab0429df" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8" is_pre_election: true
I20260929 18:40:33.752378 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb094505edf4ce0ae9d781cab0429df" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22" is_pre_election: true
I20260929 18:40:33.752503 13769 raft_consensus.cc:2454] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.752553 13909 raft_consensus.cc:2454] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:33.752923 13560 leader_election.cc:302] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd58d002bb049c59766048af325ab22, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.753125 14018 raft_consensus.cc:2781] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:33.753186 14018 raft_consensus.cc:482] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:33.753224 14018 raft_consensus.cc:3037] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.754637 14018 raft_consensus.cc:504] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.755061 14018 leader_election.cc:288] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Requested vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:33.755465 13769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb094505edf4ce0ae9d781cab0429df" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8"
I20260929 18:40:33.755511 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb094505edf4ce0ae9d781cab0429df" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22"
I20260929 18:40:33.755612 13769 raft_consensus.cc:3037] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.755647 13909 raft_consensus.cc:3037] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:33.756786 13909 raft_consensus.cc:2454] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.756808 13769 raft_consensus.cc:2454] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:33.757161 13560 leader_election.cc:302] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:33.757493 14018 raft_consensus.cc:2781] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:33.757586 14018 raft_consensus.cc:686] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Becoming Leader. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Running, Role: LEADER
I20260929 18:40:33.757720 14018 consensus_queue.cc:237] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:33.759720 13470 catalog_manager.cc:5495] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d reported cstate change: term changed from 0 to 1, leader changed from <none> to c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193). New cstate: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:33.793175 13768 raft_consensus.cc:1261] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.793721 13989 consensus_queue.cc:1035] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:33.794165 13909 raft_consensus.cc:1261] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:33.794643 13989 consensus_queue.cc:1035] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:40:33.810758 14004 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 18:40:35.367578 13470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:35.367734 13470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:35.370481 13470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Sending DeleteTablet for 3 replicas of tablet cbb094505edf4ce0ae9d781cab0429df
I20260929 18:40:35.371217 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet cbb094505edf4ce0ae9d781cab0429df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48920
I20260929 18:40:35.371398 13749 tablet_service.cc:1514] Processing DeleteTablet for tablet cbb094505edf4ce0ae9d781cab0429df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:56998
I20260929 18:40:35.371428 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet cbb094505edf4ce0ae9d781cab0429df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32946
I20260929 18:40:35.371760 14050 tablet_replica.cc:330] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: stopping tablet replica
I20260929 18:40:35.371781 14049 tablet_replica.cc:330] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: stopping tablet replica
I20260929 18:40:35.372224 14049 raft_consensus.cc:2229] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:35.372238 14050 raft_consensus.cc:2229] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:35.371767 14051 tablet_replica.cc:330] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: stopping tablet replica
I20260929 18:40:35.372442 14050 raft_consensus.cc:2258] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.372478 14051 raft_consensus.cc:2229] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:35.372673 14051 raft_consensus.cc:2258] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.372933 14049 raft_consensus.cc:2258] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.373999 14051 ts_tablet_manager.cc:1905] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.374679 14049 ts_tablet_manager.cc:1905] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.375334 14050 ts_tablet_manager.cc:1905] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.376528 14051 ts_tablet_manager.cc:1918] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 18:40:35.376641 14051 log.cc:1196] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/cbb094505edf4ce0ae9d781cab0429df
I20260929 18:40:35.376904 14051 ts_tablet_manager.cc:1939] T cbb094505edf4ce0ae9d781cab0429df P 2cd58d002bb049c59766048af325ab22: Deleting consensus metadata
I20260929 18:40:35.377601 14049 ts_tablet_manager.cc:1918] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 18:40:35.377738 14049 log.cc:1196] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/cbb094505edf4ce0ae9d781cab0429df
I20260929 18:40:35.377835 13456 catalog_manager.cc:4841] TS 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797): tablet cbb094505edf4ce0ae9d781cab0429df (table test-table [id=c7f1bad1db594de58f9a01516d0d3063]) successfully deleted
I20260929 18:40:35.378113 14049 ts_tablet_manager.cc:1939] T cbb094505edf4ce0ae9d781cab0429df P c70183fe74b942158c75ca5cb9a1c94d: Deleting consensus metadata
I20260929 18:40:35.378361 14050 ts_tablet_manager.cc:1918] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 18:40:35.378443 14050 log.cc:1196] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/cbb094505edf4ce0ae9d781cab0429df
I20260929 18:40:35.378772 14050 ts_tablet_manager.cc:1939] T cbb094505edf4ce0ae9d781cab0429df P 2489e38864c04dbe856ad181f54a64c8: Deleting consensus metadata
I20260929 18:40:35.378912 13457 catalog_manager.cc:4841] TS c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281): tablet cbb094505edf4ce0ae9d781cab0429df (table test-table [id=c7f1bad1db594de58f9a01516d0d3063]) successfully deleted
I20260929 18:40:35.379467 13456 catalog_manager.cc:4841] TS 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621): tablet cbb094505edf4ce0ae9d781cab0429df (table test-table [id=c7f1bad1db594de58f9a01516d0d3063]) successfully deleted
Sep 29 18:40:35 dist-test-slave-cgv7 krb5kdc[13427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707235, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 18:40:35 dist-test-slave-cgv7 krb5kdc[13427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707235, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:35.404093 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.395059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.11.118.254:39097)
0929 18:40:35.395314 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.395321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.395336 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:35.395456 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.395459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.395503 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.395593 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:35.396321 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.396882 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.397748 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.398003 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.398108 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.400859 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:35.400862 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:35.400864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:35.400891 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:35.402524 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:35.402932 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:35.402935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:35.402937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:35.403021 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:35.403318 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:35.403321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:35.403322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:35.403527 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:35.403636 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.403854 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.403960 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":109,"threads_started":1}
I20260929 18:40:35.407018 13470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:35.407487 13470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:35.412648 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 (DEFAULT_TABLE table=test-table [id=4871d93476e54927be6ca64dd03fb9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.412648 13889 tablet_service.cc:1467] Processing CreateTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 (DEFAULT_TABLE table=test-table [id=4871d93476e54927be6ca64dd03fb9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.412984 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a1d6ebda7d42a385bd53d49c90fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.412984 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a1d6ebda7d42a385bd53d49c90fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.414597 13749 tablet_service.cc:1467] Processing CreateTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 (DEFAULT_TABLE table=test-table [id=4871d93476e54927be6ca64dd03fb9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.414953 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a1d6ebda7d42a385bd53d49c90fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.415786 14060 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Bootstrap starting.
I20260929 18:40:35.416064 14061 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Bootstrap starting.
I20260929 18:40:35.416828 14060 tablet_bootstrap.cc:654] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.417340 14062 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Bootstrap starting.
I20260929 18:40:35.417353 14061 tablet_bootstrap.cc:654] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.418154 14060 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: No bootstrap required, opened a new log
I20260929 18:40:35.418265 14060 ts_tablet_manager.cc:1397] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:35.418383 14062 tablet_bootstrap.cc:654] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.418784 14060 raft_consensus.cc:348] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.418926 14060 raft_consensus.cc:374] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.418962 14060 raft_consensus.cc:729] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.419090 14061 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: No bootstrap required, opened a new log
I20260929 18:40:35.419174 14061 ts_tablet_manager.cc:1397] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:35.419127 14060 consensus_queue.cc:260] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.419420 14060 ts_tablet_manager.cc:1428] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:35.419631 14061 raft_consensus.cc:348] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.419721 14061 raft_consensus.cc:374] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.419742 14061 raft_consensus.cc:729] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.419890 14061 consensus_queue.cc:260] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.420156 14062 tablet_bootstrap.cc:492] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: No bootstrap required, opened a new log
I20260929 18:40:35.420244 14062 ts_tablet_manager.cc:1397] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260929 18:40:35.420358 14061 ts_tablet_manager.cc:1428] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:35.420710 14062 raft_consensus.cc:348] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.420835 14062 raft_consensus.cc:374] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.420883 14062 raft_consensus.cc:729] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2489e38864c04dbe856ad181f54a64c8, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.421021 14062 consensus_queue.cc:260] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.421413 14062 ts_tablet_manager.cc:1428] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 18:40:35.504719 13985 raft_consensus.cc:482] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:35.504868 13985 raft_consensus.cc:504] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.505388 13985 leader_election.cc:288] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:35.505980 13908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a1d6ebda7d42a385bd53d49c90fad4" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22" is_pre_election: true
I20260929 18:40:35.505959 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a1d6ebda7d42a385bd53d49c90fad4" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8" is_pre_election: true
I20260929 18:40:35.506126 13908 raft_consensus.cc:2454] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:35.506131 13768 raft_consensus.cc:2454] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 0.
I20260929 18:40:35.506512 13560 leader_election.cc:302] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd58d002bb049c59766048af325ab22, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:35.506749 13985 raft_consensus.cc:2781] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:35.506839 13985 raft_consensus.cc:482] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:35.506892 13985 raft_consensus.cc:3037] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.507925 13985 raft_consensus.cc:504] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.508340 13985 leader_election.cc:288] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Requested vote from peers 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621), 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797)
I20260929 18:40:35.508683 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a1d6ebda7d42a385bd53d49c90fad4" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8"
I20260929 18:40:35.508837 13768 raft_consensus.cc:3037] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.508836 13908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a1d6ebda7d42a385bd53d49c90fad4" candidate_uuid: "c70183fe74b942158c75ca5cb9a1c94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd58d002bb049c59766048af325ab22"
I20260929 18:40:35.508978 13908 raft_consensus.cc:3037] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.510159 13768 raft_consensus.cc:2454] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:35.510210 13908 raft_consensus.cc:2454] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70183fe74b942158c75ca5cb9a1c94d in term 1.
I20260929 18:40:35.510526 13560 leader_election.cc:302] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, c70183fe74b942158c75ca5cb9a1c94d; no voters:
I20260929 18:40:35.510819 13985 raft_consensus.cc:2781] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:35.510919 13985 raft_consensus.cc:686] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Becoming Leader. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Running, Role: LEADER
I20260929 18:40:35.511139 13985 consensus_queue.cc:237] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.512920 13469 catalog_manager.cc:5495] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d reported cstate change: term changed from 0 to 1, leader changed from <none> to c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193). New cstate: current_term: 1 leader_uuid: "c70183fe74b942158c75ca5cb9a1c94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:35.548415 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.545139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.11.118.193:33281)
0929 18:40:35.545435 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.545444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.545456 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:40:35.545499 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.545501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.545539 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.545626 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:35.545893 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.546485 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.547353 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.547586 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.547691 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.547960 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.548032 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.548112 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.548204 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.548256 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":116,"threads_started":1}
I20260929 18:40:35.555215 13908 raft_consensus.cc:1261] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:35.555270 13768 raft_consensus.cc:1261] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Refusing update from remote peer c70183fe74b942158c75ca5cb9a1c94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:35.555624 14030 consensus_queue.cc:1035] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:35.555784 13985 consensus_queue.cc:1035] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:35.587229 13471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:35.587415 13471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:35.589601 13471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Sending DeleteTablet for 3 replicas of tablet 34a1d6ebda7d42a385bd53d49c90fad4
I20260929 18:40:35.590174 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48920
I20260929 18:40:35.590252 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32946
I20260929 18:40:35.590369 13749 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a1d6ebda7d42a385bd53d49c90fad4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:35 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:56998
I20260929 18:40:35.590415 14049 tablet_replica.cc:330] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: stopping tablet replica
I20260929 18:40:35.590461 14051 tablet_replica.cc:330] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: stopping tablet replica
I20260929 18:40:35.590525 14049 raft_consensus.cc:2229] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:35.590574 14051 raft_consensus.cc:2229] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:35.590683 14051 raft_consensus.cc:2258] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.590723 14049 raft_consensus.cc:2258] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.590824 14050 tablet_replica.cc:330] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: stopping tablet replica
I20260929 18:40:35.590957 14050 raft_consensus.cc:2229] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:35.591109 14050 raft_consensus.cc:2258] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:35.591506 14049 ts_tablet_manager.cc:1905] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.591645 13471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:35.592038 13471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:35.593873 14049 ts_tablet_manager.cc:1918] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:35.593971 14049 log.cc:1196] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/34a1d6ebda7d42a385bd53d49c90fad4
I20260929 18:40:35.594235 14049 ts_tablet_manager.cc:1939] T 34a1d6ebda7d42a385bd53d49c90fad4 P c70183fe74b942158c75ca5cb9a1c94d: Deleting consensus metadata
I20260929 18:40:35.594301 14050 ts_tablet_manager.cc:1905] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.594967 13457 catalog_manager.cc:4841] TS c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281): tablet 34a1d6ebda7d42a385bd53d49c90fad4 (table test-table [id=4871d93476e54927be6ca64dd03fb9a8]) successfully deleted
I20260929 18:40:35.596910 14050 ts_tablet_manager.cc:1918] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:35.596925 14051 ts_tablet_manager.cc:1905] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:35.596999 14050 log.cc:1196] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/34a1d6ebda7d42a385bd53d49c90fad4
I20260929 18:40:35.597307 14050 ts_tablet_manager.cc:1939] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2489e38864c04dbe856ad181f54a64c8: Deleting consensus metadata
I20260929 18:40:35.598191 13456 catalog_manager.cc:4841] TS 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621): tablet 34a1d6ebda7d42a385bd53d49c90fad4 (table test-table [id=4871d93476e54927be6ca64dd03fb9a8]) successfully deleted
I20260929 18:40:35.598698 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb (DEFAULT_TABLE table=test-table [id=f2b5f3a766ee456e86728432bbeebba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.598790 13889 tablet_service.cc:1467] Processing CreateTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb (DEFAULT_TABLE table=test-table [id=f2b5f3a766ee456e86728432bbeebba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.598938 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be0b0a938e442ae8ac331799ab57bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.599017 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be0b0a938e442ae8ac331799ab57bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.599362 14051 ts_tablet_manager.cc:1918] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:40:35.599424 14051 log.cc:1196] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/34a1d6ebda7d42a385bd53d49c90fad4
I20260929 18:40:35.599668 14051 ts_tablet_manager.cc:1939] T 34a1d6ebda7d42a385bd53d49c90fad4 P 2cd58d002bb049c59766048af325ab22: Deleting consensus metadata
I20260929 18:40:35.600872 14060 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: Bootstrap starting.
I20260929 18:40:35.601209 14061 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Bootstrap starting.
I20260929 18:40:35.601586 14060 tablet_bootstrap.cc:654] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.602056 14061 tablet_bootstrap.cc:654] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.602619 14060 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: No bootstrap required, opened a new log
I20260929 18:40:35.602702 14060 ts_tablet_manager.cc:1397] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 18:40:35.603076 13749 tablet_service.cc:1467] Processing CreateTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb (DEFAULT_TABLE table=test-table [id=f2b5f3a766ee456e86728432bbeebba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:35.603096 14060 raft_consensus.cc:348] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.603183 14060 raft_consensus.cc:374] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.603204 14060 raft_consensus.cc:729] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70183fe74b942158c75ca5cb9a1c94d, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.603307 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be0b0a938e442ae8ac331799ab57bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:35.603340 14060 consensus_queue.cc:260] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.603618 14060 ts_tablet_manager.cc:1428] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:40:35.605777 14062 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Bootstrap starting.
I20260929 18:40:35.606874 14062 tablet_bootstrap.cc:654] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:35.608105 14061 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: No bootstrap required, opened a new log
I20260929 18:40:35.608198 14061 ts_tablet_manager.cc:1397] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 18:40:35.608281 14062 tablet_bootstrap.cc:492] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: No bootstrap required, opened a new log
I20260929 18:40:35.608348 14062 ts_tablet_manager.cc:1397] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 18:40:35.608621 14061 raft_consensus.cc:348] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.608726 14061 raft_consensus.cc:374] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.608754 14061 raft_consensus.cc:729] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.608741 14062 raft_consensus.cc:348] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.608825 14062 raft_consensus.cc:374] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:35.608846 14062 raft_consensus.cc:729] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2489e38864c04dbe856ad181f54a64c8, State: Initialized, Role: FOLLOWER
I20260929 18:40:35.608868 14061 consensus_queue.cc:260] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.608937 14062 consensus_queue.cc:260] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.609104 14061 ts_tablet_manager.cc:1428] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:35.609198 14062 ts_tablet_manager.cc:1428] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 18:40:35.610041 13456 catalog_manager.cc:4841] TS 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797): tablet 34a1d6ebda7d42a385bd53d49c90fad4 (table test-table [id=4871d93476e54927be6ca64dd03fb9a8]) successfully deleted
I20260929 18:40:35.700788 14066 raft_consensus.cc:482] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:35.700973 14066 raft_consensus.cc:504] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.702109 14066 leader_election.cc:288] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281), 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621)
I20260929 18:40:35.707931 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.702577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:47629 (local address 127.11.118.194:37621)
0929 18:40:35.702878 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.702890 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.703037 (+ 147us) server_negotiation.cc:407] Connection header received
0929 18:40:35.703114 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.703118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.703179 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.703268 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:35.703742 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.704581 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.706360 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.707028 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.707168 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.707222 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.707697 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.707781 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260929 18:40:35.707911 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.702569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:45307 (local address 127.11.118.193:33281)
0929 18:40:35.702744 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.702750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.702967 (+ 217us) server_negotiation.cc:407] Connection header received
0929 18:40:35.703119 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.703122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.703179 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.703268 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:35.703742 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.704581 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.706360 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.707031 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.707168 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.707222 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.707697 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.707781 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 18:40:35.708156 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.702456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:37621 (local address 127.11.118.195:47629)
0929 18:40:35.702924 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:35.702937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:35.703020 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:35.703299 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:35.703302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:35.703343 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:35.703571 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.703581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.704746 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.704751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.706208 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.706215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.707185 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.707187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.707475 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.707535 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 18:40:35.707664 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 18:40:35.707729 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":416,"spinlock_wait_cycles":32128,"thread_start_us":98,"threads_started":1}
I20260929 18:40:35.707950 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.702456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.11.118.195:45307)
0929 18:40:35.702860 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:35.702877 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:35.703010 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:35.703299 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:35.703302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:35.703344 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:35.703588 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.703593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.704746 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.704751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.706201 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.706215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.707182 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.707187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.707480 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.707534 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 18:40:35.707621 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 18:40:35.707731 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":268,"spinlock_wait_cycles":6656,"thread_start_us":106,"threads_started":1}
I20260929 18:40:35.708751 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be0b0a938e442ae8ac331799ab57bfb" candidate_uuid: "2cd58d002bb049c59766048af325ab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8" is_pre_election: true
I20260929 18:40:35.708948 13768 raft_consensus.cc:2454] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd58d002bb049c59766048af325ab22 in term 0.
I20260929 18:40:35.708922 13629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be0b0a938e442ae8ac331799ab57bfb" candidate_uuid: "2cd58d002bb049c59766048af325ab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70183fe74b942158c75ca5cb9a1c94d" is_pre_election: true
I20260929 18:40:35.709195 13629 raft_consensus.cc:2454] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd58d002bb049c59766048af325ab22 in term 0.
I20260929 18:40:35.709329 13840 leader_election.cc:302] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, 2cd58d002bb049c59766048af325ab22; no voters:
I20260929 18:40:35.709635 14066 raft_consensus.cc:2781] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:35.709698 14066 raft_consensus.cc:482] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:35.709743 14066 raft_consensus.cc:3037] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.710963 14066 raft_consensus.cc:504] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.711354 14066 leader_election.cc:288] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [CANDIDATE]: Term 1 election: Requested vote from peers c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281), 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621)
I20260929 18:40:35.711683 13629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be0b0a938e442ae8ac331799ab57bfb" candidate_uuid: "2cd58d002bb049c59766048af325ab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70183fe74b942158c75ca5cb9a1c94d"
I20260929 18:40:35.711692 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be0b0a938e442ae8ac331799ab57bfb" candidate_uuid: "2cd58d002bb049c59766048af325ab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2489e38864c04dbe856ad181f54a64c8"
I20260929 18:40:35.711786 13629 raft_consensus.cc:3037] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.711817 13768 raft_consensus.cc:3037] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:35.712718 13768 raft_consensus.cc:2454] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd58d002bb049c59766048af325ab22 in term 1.
I20260929 18:40:35.712790 13629 raft_consensus.cc:2454] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd58d002bb049c59766048af325ab22 in term 1.
I20260929 18:40:35.713068 13840 leader_election.cc:302] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2489e38864c04dbe856ad181f54a64c8, 2cd58d002bb049c59766048af325ab22; no voters:
I20260929 18:40:35.713228 14066 raft_consensus.cc:2781] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:35.713474 14066 raft_consensus.cc:686] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd58d002bb049c59766048af325ab22, State: Running, Role: LEADER
I20260929 18:40:35.713770 14066 consensus_queue.cc:237] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } }
I20260929 18:40:35.716223 13470 catalog_manager.cc:5495] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd58d002bb049c59766048af325ab22 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "2cd58d002bb049c59766048af325ab22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd58d002bb049c59766048af325ab22" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39797 } health_report { overall_health: HEALTHY } } }
I20260929 18:40:35.748523 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.744843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.11.118.195:39797)
0929 18:40:35.745132 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.745144 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.745155 (+ 11us) server_negotiation.cc:407] Connection header received
0929 18:40:35.745224 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.745226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.745275 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.745397 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:35.745636 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.746189 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.747039 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.747280 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.747384 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.747657 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.748099 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.748220 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.748286 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.748337 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":107,"threads_started":1}
I20260929 18:40:35.758607 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.755138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:59735 (local address 127.11.118.193:33281)
0929 18:40:35.755285 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.755289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.755507 (+ 218us) server_negotiation.cc:407] Connection header received
0929 18:40:35.755588 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.755590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.755638 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.755709 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:35.755976 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.756464 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.757468 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.757685 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.757772 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.758142 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.758212 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.758287 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.758447 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.758497 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 18:40:35.758621 14085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.755007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.11.118.195:59735)
0929 18:40:35.755408 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:35.755420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:35.755518 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:35.755735 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:35.755737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:35.755743 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:35.755840 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.755844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.756584 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.756587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.757341 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.757351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.757842 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.757845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.757963 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.758008 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.758353 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.758359 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 18:40:35.758420 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 18:40:35.758486 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":119,"threads_started":1}
I20260929 18:40:35.764623 13768 raft_consensus.cc:1261] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Refusing update from remote peer 2cd58d002bb049c59766048af325ab22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:35.764820 13629 raft_consensus.cc:1261] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d [term 1 FOLLOWER]: Refusing update from remote peer 2cd58d002bb049c59766048af325ab22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:35.765161 14080 consensus_queue.cc:1035] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2489e38864c04dbe856ad181f54a64c8" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:35.765328 14066 consensus_queue.cc:1035] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70183fe74b942158c75ca5cb9a1c94d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:35.783581 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.780051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:42749 (local address 127.11.118.195:39797)
0929 18:40:35.780183 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:35.780188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:35.780340 (+ 152us) server_negotiation.cc:407] Connection header received
0929 18:40:35.780512 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:35.780515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:35.780557 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:35.780629 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:35.780791 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.781282 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.782353 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.782638 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.782765 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.783167 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.783255 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.783333 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:35.783406 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 18:40:35.783452 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 18:40:35.783607 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:35.779949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:39797 (local address 127.11.118.193:42749)
0929 18:40:35.780269 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:35.780284 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:35.780386 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:35.780634 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:35.780635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:35.780642 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:40:35.780731 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.780734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.781368 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.781370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.782208 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:35.782222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:35.782847 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:35.782850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:35.782949 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:35.782978 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:40:35.783340 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:40:35.783346 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 18:40:35.783397 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 18:40:35.783470 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":126,"threads_started":1}
W20260929 18:40:35.784330 14034 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 18:40:37.345316 13469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:37.345484 13469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:37.348147 13469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a37efa6e3c5549dfa532fc2feb30dcc4: Sending DeleteTablet for 3 replicas of tablet 7be0b0a938e442ae8ac331799ab57bfb
I20260929 18:40:37.348878 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:37 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48920
I20260929 18:40:37.348898 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:37 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32946
I20260929 18:40:37.349016 13749 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be0b0a938e442ae8ac331799ab57bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:37 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:56998
I20260929 18:40:37.349408 14116 tablet_replica.cc:330] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: stopping tablet replica
I20260929 18:40:37.349408 14114 tablet_replica.cc:330] T 7be0b0a938e442ae8ac331799ab57bfb P c70183fe74b942158c75ca5cb9a1c94d: stopping tablet replica
I20260929 18:40:37.349493 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13544
I20260929 18:40:37.349556 14116 raft_consensus.cc:2229] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:37.351226 14116 raft_consensus.cc:2258] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:37.351877 14116 ts_tablet_manager.cc:1905] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:37.352757 14115 tablet_replica.cc:330] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: stopping tablet replica
I20260929 18:40:37.352921 14115 raft_consensus.cc:2229] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:37.353083 14115 raft_consensus.cc:2258] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:37.353742 14115 ts_tablet_manager.cc:1905] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:37.354982 14116 ts_tablet_manager.cc:1918] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 18:40:37.355252 14116 log.cc:1196] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/7be0b0a938e442ae8ac331799ab57bfb
I20260929 18:40:37.355615 14116 ts_tablet_manager.cc:1939] T 7be0b0a938e442ae8ac331799ab57bfb P 2cd58d002bb049c59766048af325ab22: Deleting consensus metadata
I20260929 18:40:37.356460 13456 catalog_manager.cc:4841] TS 2cd58d002bb049c59766048af325ab22 (127.11.118.195:39797): tablet 7be0b0a938e442ae8ac331799ab57bfb (table test-table [id=f2b5f3a766ee456e86728432bbeebba4]) successfully deleted
I20260929 18:40:37.356909 14115 ts_tablet_manager.cc:1918] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 18:40:37.356971 14115 log.cc:1196] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/7be0b0a938e442ae8ac331799ab57bfb
I20260929 18:40:37.357231 14115 ts_tablet_manager.cc:1939] T 7be0b0a938e442ae8ac331799ab57bfb P 2489e38864c04dbe856ad181f54a64c8: Deleting consensus metadata
I20260929 18:40:37.357924 13456 catalog_manager.cc:4841] TS 2489e38864c04dbe856ad181f54a64c8 (127.11.118.194:37621): tablet 7be0b0a938e442ae8ac331799ab57bfb (table test-table [id=f2b5f3a766ee456e86728432bbeebba4]) successfully deleted
W20260929 18:40:37.359058 13457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:33281 (error 108)
I20260929 18:40:37.359120 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13684
I20260929 18:40:37.360893 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:37.360103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:33281 (local address 127.0.0.1:48942)
0929 18:40:37.360748 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:37.360811 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:33281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":77,"threads_started":1}
W20260929 18:40:37.361035 13457 catalog_manager.cc:4584] TS c70183fe74b942158c75ca5cb9a1c94d (127.11.118.193:33281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7be0b0a938e442ae8ac331799ab57bfb: Network error: Client connection negotiation failed: client connection to 127.11.118.193:33281: connect: Connection refused (error 111)
I20260929 18:40:37.365279 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13824
I20260929 18:40:37.372486 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 13440
2026-09-29T18:40:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6475 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:37 dist-test-slave-cgv7 krb5kdc[14121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:37 dist-test-slave-cgv7 krb5kdc[14121](info): set up 2 sockets
Sep 29 18:40:37 dist-test-slave-cgv7 krb5kdc[14121](info): commencing operation
krb5kdc: starting...
W20260929 18:40:38.430557 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:38 dist-test-slave-cgv7 krb5kdc[14121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:38.553730 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38927
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:38927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:38.665601 14134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:38.665926 14134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:38.665969 14134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:38.670378 14134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:38.670446 14134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:38.670466 14134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:38.670483 14134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:38.670527 14134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:38.676003 14134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:38927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:38.677131 14134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:38.678180 14134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:38.687278 14134 server_base.cc:1034] running on GCE node
W20260929 18:40:38.687310 14140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:38.687383 14139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:38.687557 14142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:38.688126 14134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:38.689318 14134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:38.690490 14134 hybrid_clock.cc:648] HybridClock initialized: now 1790707238690475 us; error 31 us; skew 500 ppm
Sep 29 18:40:38 dist-test-slave-cgv7 krb5kdc[14121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707238, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:38.693406 14134 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:38.694418 14134 webserver.cc:460] Webserver started at http://127.11.118.254:39435/ using document root <none> and password file <none>
I20260929 18:40:38.695004 14134 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:38.695068 14134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:38.695297 14134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:38.697350 14134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "dfaa95f6e75343b082f32b15557ee80e"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "76a3799445899a843d03b096b0c98eda"
server_key_iv: "dbdad69d6261a1b4a3354da4939d86c9"
server_key_version: "encryptionkey@0"
I20260929 18:40:38.697939 14134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "dfaa95f6e75343b082f32b15557ee80e"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "76a3799445899a843d03b096b0c98eda"
server_key_iv: "dbdad69d6261a1b4a3354da4939d86c9"
server_key_version: "encryptionkey@0"
I20260929 18:40:38.701785 14134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 18:40:38.704326 14149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:38.705732 14134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:38.705859 14134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "dfaa95f6e75343b082f32b15557ee80e"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "76a3799445899a843d03b096b0c98eda"
server_key_iv: "dbdad69d6261a1b4a3354da4939d86c9"
server_key_version: "encryptionkey@0"
I20260929 18:40:38.705974 14134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:38.716581 14134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:38.719738 14134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:38.719942 14134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:38.727448 14134 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:38927
I20260929 18:40:38.727471 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:38927 every 8 connection(s)
I20260929 18:40:38.728534 14134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:38.730106 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14134
I20260929 18:40:38.730232 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:38.730603 11739 external_mini_cluster.cc:1428] Setting key 5c8953be6fa3b0ae17299abc9ae3a4f0
I20260929 18:40:38.732173 14202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:38.737987 14202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e: Bootstrap starting.
Sep 29 18:40:38 dist-test-slave-cgv7 krb5kdc[14121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:38.740852 14202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:38.741856 14202 log.cc:824] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:38.744223 14202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e: No bootstrap required, opened a new log
I20260929 18:40:38.745968 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:38.732089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.11.118.254:38927)
0929 18:40:38.732549 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:38.732565 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:38.732602 (+ 37us) server_negotiation.cc:407] Connection header received
0929 18:40:38.733253 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:38.733269 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:38.733615 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:38.733962 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:38.735009 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:38.736018 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:38.736884 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:38.737386 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:38.737560 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:38.740494 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:38.740500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:38.740516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:38.740549 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:38.742847 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:38.743389 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:38.743395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:38.743400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:38.743517 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:38.743853 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:38.743857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:38.743858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:38.744340 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:38.744573 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:38.744865 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 18:40:38.745179 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":149,"threads_started":1}
I20260929 18:40:38.747989 14202 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } }
I20260929 18:40:38.748308 14202 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:38.748394 14202 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfaa95f6e75343b082f32b15557ee80e, State: Initialized, Role: FOLLOWER
I20260929 18:40:38.748795 14202 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } }
I20260929 18:40:38.748931 14202 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:38.749001 14202 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:38.749100 14202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:38.750056 14202 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } }
I20260929 18:40:38.750483 14202 leader_election.cc:302] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfaa95f6e75343b082f32b15557ee80e; no voters:
I20260929 18:40:38.750834 14202 leader_election.cc:288] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:38.750972 14207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:38.751231 14207 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [term 1 LEADER]: Becoming Leader. State: Replica: dfaa95f6e75343b082f32b15557ee80e, State: Running, Role: LEADER
I20260929 18:40:38.751582 14207 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } }
I20260929 18:40:38.752292 14202 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:38.753525 14209 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfaa95f6e75343b082f32b15557ee80e. Latest consensus state: current_term: 1 leader_uuid: "dfaa95f6e75343b082f32b15557ee80e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } } }
I20260929 18:40:38.753573 14208 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfaa95f6e75343b082f32b15557ee80e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaa95f6e75343b082f32b15557ee80e" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38927 } } }
I20260929 18:40:38.753664 14209 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:38.753695 14208 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:38.754016 14216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:38.756937 14216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:38.763638 14216 catalog_manager.cc:1347] Generated new cluster ID: 2ca644743d0a40a09d32c82312095e30
I20260929 18:40:38.763904 14216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:38.798005 14216 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:38.799474 14216 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:38.810768 14216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfaa95f6e75343b082f32b15557ee80e: Generated new TSK 0
I20260929 18:40:38.811688 14216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:38.858105 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:38.972023 14228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:38.972335 14228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:38.972385 14228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:38.976583 14228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:38.976645 14228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:38.976706 14228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:38.982076 14228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:38.983239 14228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:38.984284 14228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:38.994256 14236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:38.994256 14234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:38.994318 14228 server_base.cc:1034] running on GCE node
W20260929 18:40:38.994272 14233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:38.994943 14228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:38.995589 14228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:38.996798 14228 hybrid_clock.cc:648] HybridClock initialized: now 1790707238996779 us; error 31 us; skew 500 ppm
Sep 29 18:40:38 dist-test-slave-cgv7 krb5kdc[14121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707238, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:38.999720 14228 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:39.000617 14228 webserver.cc:460] Webserver started at http://127.11.118.193:42705/ using document root <none> and password file <none>
I20260929 18:40:39.001152 14228 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:39.001214 14228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:39.001374 14228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:39.003432 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "a3f44efc31774054bbbae8cea227fdf6"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "4ea2012de6a91d3c2d50ab1c51655dd6"
server_key_iv: "dfbd02f0f3fbc5c6f3e35f49911a0699"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.003965 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "a3f44efc31774054bbbae8cea227fdf6"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "4ea2012de6a91d3c2d50ab1c51655dd6"
server_key_iv: "dfbd02f0f3fbc5c6f3e35f49911a0699"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.007622 14228 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:40:39.010224 14243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.011690 14228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:39.011796 14228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "a3f44efc31774054bbbae8cea227fdf6"
format_stamp: "Formatted at 2026-09-29 18:40:38 on dist-test-slave-cgv7"
server_key: "4ea2012de6a91d3c2d50ab1c51655dd6"
server_key_iv: "dfbd02f0f3fbc5c6f3e35f49911a0699"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.011885 14228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:39.031878 14228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:39.035450 14228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:39.035617 14228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:39.036187 14228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:39.037060 14228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:39.037110 14228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.037151 14228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:39.037168 14228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.046882 14228 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45275
I20260929 18:40:39.046969 14356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45275 every 8 connection(s)
I20260929 18:40:39.047823 14228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:39.055226 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14228
I20260929 18:40:39.055323 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:39.055598 11739 external_mini_cluster.cc:1428] Setting key 64882b07cc833716077a81367b4f77fc
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707238, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:39.064594 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.049779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:38659 (local address 127.11.118.254:38927)
0929 18:40:39.049923 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:39.049926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:39.051138 (+ 1212us) server_negotiation.cc:407] Connection header received
0929 18:40:39.052141 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:39.052144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:39.052205 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:39.052319 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:39.053616 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.054183 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.055463 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.055737 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.055849 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.060075 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:39.060079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:39.060081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:39.060106 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:39.062040 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.062791 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.062798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.062803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.062910 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.063257 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.063264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.063268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.063564 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:39.063684 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:39.064235 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 18:40:39.064379 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:40:39.065392 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.050194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38927 (local address 127.11.118.193:38659)
0929 18:40:39.050950 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:39.051016 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:39.051927 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:39.052489 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:39.052498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:39.052937 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:39.053467 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.053482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.054327 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.054336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.055264 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.055274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.055904 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.055908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.056085 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.057772 (+ 1687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:39.057799 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:39.059885 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:39.062176 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.062182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.062197 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.062657 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.063011 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.063013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.063014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.063150 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.063676 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:39.063681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:39.063963 (+ 282us) client_negotiation.cc:769] Sending connection context
0929 18:40:39.064122 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 18:40:39.064388 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:39.066955 14357 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38927
I20260929 18:40:39.067260 14357 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:39.067806 14357 heartbeater.cc:511] Master 127.11.118.254:38927 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:39.069465 14166 ts_manager.cc:194] Registered new tserver with Master: a3f44efc31774054bbbae8cea227fdf6 (127.11.118.193:45275)
I20260929 18:40:39.070693 14166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:38659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:39.085225 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:39.202442 14362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:39.202762 14362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:39.202854 14362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:39.206977 14362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:39.207072 14362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:39.207165 14362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:39.212787 14362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:39.213946 14362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:39.215018 14362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:39.225271 14367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:39.225271 14368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:39.225421 14362 server_base.cc:1034] running on GCE node
W20260929 18:40:39.225514 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:39.226146 14362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:39.226845 14362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:39.228096 14362 hybrid_clock.cc:648] HybridClock initialized: now 1790707239228083 us; error 37 us; skew 500 ppm
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:39.231385 14362 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:39.232419 14362 webserver.cc:460] Webserver started at http://127.11.118.194:39571/ using document root <none> and password file <none>
I20260929 18:40:39.233131 14362 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:39.233240 14362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:39.233496 14362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:39.235796 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "f22fcbfb2ebd4ac6809235cec724b6cf"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "3e51ae7bc8e9067f773404202443587b"
server_key_iv: "af6b8e64aa20633062622dc49046b207"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.236440 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "f22fcbfb2ebd4ac6809235cec724b6cf"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "3e51ae7bc8e9067f773404202443587b"
server_key_iv: "af6b8e64aa20633062622dc49046b207"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.240525 14362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:39.243434 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.244827 14362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:39.245033 14362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "f22fcbfb2ebd4ac6809235cec724b6cf"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "3e51ae7bc8e9067f773404202443587b"
server_key_iv: "af6b8e64aa20633062622dc49046b207"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.245172 14362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:39.265305 14362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:39.268977 14362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:39.269197 14362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:39.269974 14362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:39.271167 14362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:39.271242 14362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.271308 14362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:39.271347 14362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.282706 14362 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:46119
I20260929 18:40:39.282725 14490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:46119 every 8 connection(s)
I20260929 18:40:39.283659 14362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:39.292470 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14362
I20260929 18:40:39.292618 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:39.292968 11739 external_mini_cluster.cc:1428] Setting key 147b8451e2c32c555d1e2e0a0e697251
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:39.298204 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.285573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:57865 (local address 127.11.118.254:38927)
0929 18:40:39.285707 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:39.285710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:39.286836 (+ 1126us) server_negotiation.cc:407] Connection header received
0929 18:40:39.287757 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:39.287760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:39.287810 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:39.287896 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:39.289145 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.289682 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.291015 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.291249 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.291346 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.294071 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:39.294075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:39.294077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:39.294101 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:39.295803 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.296497 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.296503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.296505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.296605 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.296990 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.296997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.296999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.297194 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:39.297304 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:39.297829 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 18:40:39.297961 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:40:39.299197 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.285952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38927 (local address 127.11.118.194:57865)
0929 18:40:39.286701 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:39.286743 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:39.287544 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:39.288088 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:39.288096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:39.288469 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:39.288977 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.288992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.289813 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.289816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.290857 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.290866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.291405 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.291408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.291566 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.292263 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:39.292281 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:39.293906 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:39.295938 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.295947 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.295966 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.296363 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.296714 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.296717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.296719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.296891 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.297301 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:39.297309 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:39.297587 (+ 278us) client_negotiation.cc:769] Sending connection context
0929 18:40:39.297788 (+ 201us) client_negotiation.cc:240] Negotiation successful
0929 18:40:39.298255 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":121,"threads_started":1}
I20260929 18:40:39.300505 14491 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38927
I20260929 18:40:39.300794 14491 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:39.301458 14491 heartbeater.cc:511] Master 127.11.118.254:38927 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:39.302778 14166 ts_manager.cc:194] Registered new tserver with Master: f22fcbfb2ebd4ac6809235cec724b6cf (127.11.118.194:46119)
I20260929 18:40:39.303373 14166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:57865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:39.322387 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:39.437398 14496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:39.437692 14496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:39.437737 14496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:39.441960 14496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:39.442040 14496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:39.442102 14496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:39.447634 14496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:39.448807 14496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:39.449884 14496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:39.460055 14501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:39.460054 14502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:39.460224 14496 server_base.cc:1034] running on GCE node
W20260929 18:40:39.460047 14504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:39.460803 14496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:39.461443 14496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:39.462597 14496 hybrid_clock.cc:648] HybridClock initialized: now 1790707239462566 us; error 45 us; skew 500 ppm
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:39.465426 14496 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:39.466356 14496 webserver.cc:460] Webserver started at http://127.11.118.195:43129/ using document root <none> and password file <none>
I20260929 18:40:39.466921 14496 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:39.467021 14496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:39.467254 14496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:39.469391 14496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "d2d8b929b8444a998dd1b1020eca733d"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "e3a7fc877c9edba7da324ed81a928c7a"
server_key_iv: "2bfd769cef73d345aa365bd3e0388f74"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.469955 14496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "d2d8b929b8444a998dd1b1020eca733d"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "e3a7fc877c9edba7da324ed81a928c7a"
server_key_iv: "2bfd769cef73d345aa365bd3e0388f74"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.473742 14496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 18:40:39.476490 14511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.477936 14496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:39.478107 14496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "d2d8b929b8444a998dd1b1020eca733d"
format_stamp: "Formatted at 2026-09-29 18:40:39 on dist-test-slave-cgv7"
server_key: "e3a7fc877c9edba7da324ed81a928c7a"
server_key_iv: "2bfd769cef73d345aa365bd3e0388f74"
server_key_version: "encryptionkey@0"
I20260929 18:40:39.478233 14496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:39.490157 14496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:39.493609 14496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:39.493830 14496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:39.494477 14496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:39.495348 14496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:39.495424 14496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.495496 14496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:39.495539 14496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:39.504508 14496 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:34447
I20260929 18:40:39.504575 14624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:34447 every 8 connection(s)
I20260929 18:40:39.505431 14496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:39.509229 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14496
I20260929 18:40:39.509325 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:39.509683 11739 external_mini_cluster.cc:1428] Setting key c98dd6ad56b4f18df01864f230b8a650
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:39.519814 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.507294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:38001 (local address 127.11.118.254:38927)
0929 18:40:39.507459 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:39.507462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:39.508505 (+ 1043us) server_negotiation.cc:407] Connection header received
0929 18:40:39.509539 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:39.509543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:39.509602 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:39.509701 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:39.511211 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.511747 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.512968 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.513193 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.513310 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.516064 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:39.516067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:39.516071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:39.516106 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:39.517610 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.518204 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.518208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.518210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.518343 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:39.518692 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:39.518697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:39.518699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:39.518895 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:39.519005 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:39.519519 (+ 514us) server_negotiation.cc:299] Negotiation successful
0929 18:40:39.519647 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:40:39.520731 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:39.507690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38927 (local address 127.11.118.195:38001)
0929 18:40:39.508340 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:39.508398 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:39.509247 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:39.509874 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:39.509889 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:39.510469 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:39.511080 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.511101 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.511883 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.511887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.512826 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:39.512836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:39.513351 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:39.513355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:39.513551 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:39.514184 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:39.514202 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:39.515881 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:39.517764 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.517770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.517786 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.518091 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.518444 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:39.518449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:39.518451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:39.518596 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:39.519005 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:39.519013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:39.519274 (+ 261us) client_negotiation.cc:769] Sending connection context
0929 18:40:39.519471 (+ 197us) client_negotiation.cc:240] Negotiation successful
0929 18:40:39.519802 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":109,"threads_started":1}
I20260929 18:40:39.521979 14625 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38927
I20260929 18:40:39.522238 14625 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:39.522903 14625 heartbeater.cc:511] Master 127.11.118.254:38927 requested a full tablet report, sending...
I20260929 18:40:39.523962 14166 ts_manager.cc:194] Registered new tserver with Master: d2d8b929b8444a998dd1b1020eca733d (127.11.118.195:34447)
I20260929 18:40:39.524535 14166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:38001
I20260929 18:40:39.536422 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 18:40:39.550719 14205 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:39.548627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.11.118.254:38927)
0929 18:40:39.548810 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:39.548816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:39.549315 (+ 499us) server_negotiation.cc:407] Connection header received
0929 18:40:39.549457 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:39.549460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:39.549513 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:39.549629 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:39.550590 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51124: BlockingRecv error: recv got EOF from 127.0.0.1:51124 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 18:40:39.552028 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14228
I20260929 18:40:39.557596 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14362
I20260929 18:40:39.562794 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14496
I20260929 18:40:39.568450 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14134
2026-09-29T18:40:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2195 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14637](info): set up 2 sockets
Sep 29 18:40:39 dist-test-slave-cgv7 krb5kdc[14637](info): commencing operation
krb5kdc: starting...
W20260929 18:40:40.618470 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:40 dist-test-slave-cgv7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:40.746126 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40149
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:40149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:40.859508 14650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:40.859855 14650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:40.859942 14650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:40.864320 14650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:40.864419 14650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:40.864457 14650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:40.864524 14650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:40.864564 14650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:40.870128 14650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:40149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:40.871377 14650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:40.872522 14650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:40.881585 14656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:40.881585 14655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:40.881839 14650 server_base.cc:1034] running on GCE node
W20260929 18:40:40.881585 14658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:40.882573 14650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:40.883807 14650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:40.885002 14650 hybrid_clock.cc:648] HybridClock initialized: now 1790707240884973 us; error 45 us; skew 500 ppm
Sep 29 18:40:40 dist-test-slave-cgv7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:40.887948 14650 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:40.888913 14650 webserver.cc:460] Webserver started at http://127.11.118.254:43907/ using document root <none> and password file <none>
I20260929 18:40:40.889499 14650 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:40.889596 14650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:40.889842 14650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:40.892012 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "b4a319b6a8b349978f6794eeea6af9b5"
format_stamp: "Formatted at 2026-09-29 18:40:40 on dist-test-slave-cgv7"
server_key: "db133d7c558dcb90ff855aded6f19750"
server_key_iv: "f566d80bfa56c054b668d249a8a4c459"
server_key_version: "encryptionkey@0"
I20260929 18:40:40.892633 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "b4a319b6a8b349978f6794eeea6af9b5"
format_stamp: "Formatted at 2026-09-29 18:40:40 on dist-test-slave-cgv7"
server_key: "db133d7c558dcb90ff855aded6f19750"
server_key_iv: "f566d80bfa56c054b668d249a8a4c459"
server_key_version: "encryptionkey@0"
I20260929 18:40:40.896685 14650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260929 18:40:40.899493 14665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:40.900910 14650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:40.901110 14650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "b4a319b6a8b349978f6794eeea6af9b5"
format_stamp: "Formatted at 2026-09-29 18:40:40 on dist-test-slave-cgv7"
server_key: "db133d7c558dcb90ff855aded6f19750"
server_key_iv: "f566d80bfa56c054b668d249a8a4c459"
server_key_version: "encryptionkey@0"
I20260929 18:40:40.901247 14650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:40.919443 14650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:40.922757 14650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:40.922989 14650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:40.930163 14650 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:40149
I20260929 18:40:40.930500 14717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:40149 every 8 connection(s)
I20260929 18:40:40.931205 14650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:40.933090 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14650
I20260929 18:40:40.933213 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:40.933525 11739 external_mini_cluster.cc:1428] Setting key f13917567fa7e1bad5af70f4fcdbbd7a
I20260929 18:40:40.934460 14718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:40.941396 14718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: Bootstrap starting.
Sep 29 18:40:40 dist-test-slave-cgv7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:40.944135 14718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:40.944996 14718 log.cc:824] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:40.947620 14718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: No bootstrap required, opened a new log
I20260929 18:40:40.948848 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:40.934917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.11.118.254:40149)
0929 18:40:40.935362 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:40.935377 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:40.935414 (+ 37us) server_negotiation.cc:407] Connection header received
0929 18:40:40.936061 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:40.936079 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:40.936430 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:40.936718 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:40.937700 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:40.938874 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:40.940145 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:40.940626 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:40.940750 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:40.943702 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:40.943707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:40.943721 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:40.943752 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:40.945689 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:40.946247 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:40.946252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:40.946257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:40.946441 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:40.946735 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:40.946738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:40.946739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:40.947172 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:40.947324 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:40.947651 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 18:40:40.948105 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":148,"threads_started":1}
I20260929 18:40:40.951407 14718 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } }
I20260929 18:40:40.951630 14718 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:40.951690 14718 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a319b6a8b349978f6794eeea6af9b5, State: Initialized, Role: FOLLOWER
I20260929 18:40:40.951884 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:40:40.952180 14718 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } }
I20260929 18:40:40.952337 14718 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:40.952426 14718 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:40.952549 14718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:40.953977 14718 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } }
I20260929 18:40:40.954402 14718 leader_election.cc:302] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4a319b6a8b349978f6794eeea6af9b5; no voters:
I20260929 18:40:40.954741 14718 leader_election.cc:288] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:40.954888 14724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:40.955245 14724 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [term 1 LEADER]: Becoming Leader. State: Replica: b4a319b6a8b349978f6794eeea6af9b5, State: Running, Role: LEADER
I20260929 18:40:40.955574 14724 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } }
I20260929 18:40:40.956080 14718 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:40.956830 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4a319b6a8b349978f6794eeea6af9b5. Latest consensus state: current_term: 1 leader_uuid: "b4a319b6a8b349978f6794eeea6af9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } } }
I20260929 18:40:40.956979 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:40.957396 14725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4a319b6a8b349978f6794eeea6af9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4a319b6a8b349978f6794eeea6af9b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40149 } } }
I20260929 18:40:40.957551 14725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5 [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:40.961037 14739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:40.961140 14739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:40.961242 14733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:40.961923 14733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:40.968398 14733 catalog_manager.cc:1347] Generated new cluster ID: dcc713d2d8be4cc084ed08452c80d04d
I20260929 18:40:40.968513 14733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:40.972695 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:40.966390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.11.118.254:40149)
0929 18:40:40.966511 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:40.966515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:40.967198 (+ 683us) server_negotiation.cc:407] Connection header received
0929 18:40:40.967334 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:40.967337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:40.967421 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:40.967516 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:40.968184 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:40.969011 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:40.970251 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:40.970641 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:40.970739 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:40.971704 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:40.971708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:40.971710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:40.972234 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:40.972265 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:40.972282 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:40.972361 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:40.972494 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 18:40:40.972553 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 18:40:40.985535 14733 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:40.986826 14733 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:40.993474 14733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4a319b6a8b349978f6794eeea6af9b5: Generated new TSK 0
I20260929 18:40:40.994215 14733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:40:41.005125 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14650
2026-09-29T18:40:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1433 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:41 dist-test-slave-cgv7 krb5kdc[14751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:41 dist-test-slave-cgv7 krb5kdc[14751](info): set up 2 sockets
Sep 29 18:40:41 dist-test-slave-cgv7 krb5kdc[14751](info): commencing operation
krb5kdc: starting...
W20260929 18:40:42.074481 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:42.199632 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39065
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:42.309779 14764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:42.310067 14764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:42.310107 14764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:42.314213 14764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:42.314312 14764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:42.314340 14764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:42.314357 14764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:42.314373 14764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:42.319754 14764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:42.320881 14764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:42.321925 14764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:42.331003 14769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:42.330994 14770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:42.331053 14764 server_base.cc:1034] running on GCE node
W20260929 18:40:42.330999 14772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:42.331863 14764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:42.333081 14764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:42.334312 14764 hybrid_clock.cc:648] HybridClock initialized: now 1790707242334245 us; error 44 us; skew 500 ppm
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:42.337128 14764 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:42.338030 14764 webserver.cc:460] Webserver started at http://127.11.118.254:38117/ using document root <none> and password file <none>
I20260929 18:40:42.338613 14764 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:42.338675 14764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:42.338836 14764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:42.340770 14764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "27600d8283f001768367a92158534768"
server_key_iv: "ab29ca4414e19ad370b861d1b483f44e"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.341285 14764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "27600d8283f001768367a92158534768"
server_key_iv: "ab29ca4414e19ad370b861d1b483f44e"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.344905 14764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260929 18:40:42.347409 14779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.348675 14764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:40:42.348780 14764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "27600d8283f001768367a92158534768"
server_key_iv: "ab29ca4414e19ad370b861d1b483f44e"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.348870 14764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:42.372865 14764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:42.376772 14764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:42.376964 14764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:42.384212 14764 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39065
I20260929 18:40:42.384498 14831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39065 every 8 connection(s)
I20260929 18:40:42.385309 14764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:42.386260 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14764
I20260929 18:40:42.386416 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:42.386726 11739 external_mini_cluster.cc:1428] Setting key 0d4a27a8a9da2b5ca94d830b72796d42
I20260929 18:40:42.388801 14832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:42.395785 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: Bootstrap starting.
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:42.398692 14832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:42.399581 14832 log.cc:824] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:42.401759 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: No bootstrap required, opened a new log
I20260929 18:40:42.402124 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:42.388161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55632 (local address 127.11.118.254:39065)
0929 18:40:42.388565 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:42.388591 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:42.388626 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:40:42.389330 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:42.389349 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:42.389681 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:42.389976 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:42.390925 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.391971 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.393151 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.393640 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.393765 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:42.396767 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:42.396773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:42.396790 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:42.396826 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:42.398838 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.399337 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.399346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.399353 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.399504 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.399814 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.399817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.399818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.400367 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:42.400540 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:42.400945 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 18:40:42.401295 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":144,"threads_started":1}
I20260929 18:40:42.405753 14832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } }
I20260929 18:40:42.406020 14832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:42.406100 14832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b7149f17a04cb3a6c5f34f0cd95ac5, State: Initialized, Role: FOLLOWER
I20260929 18:40:42.406595 14832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } }
I20260929 18:40:42.406751 14832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:42.406837 14832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:42.406951 14832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:42.407907 14832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } }
I20260929 18:40:42.408250 14832 leader_election.cc:302] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23b7149f17a04cb3a6c5f34f0cd95ac5; no voters:
I20260929 18:40:42.408597 14832 leader_election.cc:288] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:42.408759 14837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:42.409338 14837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [term 1 LEADER]: Becoming Leader. State: Replica: 23b7149f17a04cb3a6c5f34f0cd95ac5, State: Running, Role: LEADER
I20260929 18:40:42.409696 14837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } }
I20260929 18:40:42.409976 14832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:42.411444 14839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23b7149f17a04cb3a6c5f34f0cd95ac5. Latest consensus state: current_term: 1 leader_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } } }
I20260929 18:40:42.411631 14839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:42.412920 14838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b7149f17a04cb3a6c5f34f0cd95ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39065 } } }
I20260929 18:40:42.413069 14838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5 [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:42.414983 14852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:42.415045 14852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:42.415123 14848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:42.415802 14848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:42.421736 14848 catalog_manager.cc:1347] Generated new cluster ID: 1efb6fc80fcf470a89b084fa184b6161
I20260929 18:40:42.421815 14848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:42.435220 14848 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:42.436174 14848 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:42.444274 14848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23b7149f17a04cb3a6c5f34f0cd95ac5: Generated new TSK 0
I20260929 18:40:42.445076 14848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:42.478088 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:42.609406 14858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:42.609756 14858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:42.609807 14858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:42.614298 14858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:42.614375 14858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:42.614490 14858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:42.619966 14858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:42.621025 14858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:42.622009 14858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:42.632395 14858 server_base.cc:1034] running on GCE node
W20260929 18:40:42.632309 14866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:42.632304 14863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:42.632309 14864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:42.633159 14858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:42.633786 14858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:42.634944 14858 hybrid_clock.cc:648] HybridClock initialized: now 1790707242634905 us; error 53 us; skew 500 ppm
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:42.641420 14858 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:42.642426 14858 webserver.cc:460] Webserver started at http://127.11.118.193:46823/ using document root <none> and password file <none>
I20260929 18:40:42.643016 14858 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:42.643110 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:42.643345 14858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:42.645429 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "3e88ec72d4554cd39b539b40bc2075ef"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "b1e6a1d7d3303e922219faaec0694372"
server_key_iv: "08c9c4ae935c4b594644814da3a56343"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.646014 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "3e88ec72d4554cd39b539b40bc2075ef"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "b1e6a1d7d3303e922219faaec0694372"
server_key_iv: "08c9c4ae935c4b594644814da3a56343"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.650012 14858 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 18:40:42.652796 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.654503 14858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:42.654640 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "3e88ec72d4554cd39b539b40bc2075ef"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "b1e6a1d7d3303e922219faaec0694372"
server_key_iv: "08c9c4ae935c4b594644814da3a56343"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.654755 14858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:42.668745 14858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:42.671787 14858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:42.672000 14858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:42.672600 14858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:42.673450 14858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:42.673496 14858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.673564 14858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:42.673596 14858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.682880 14858 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:34485
I20260929 18:40:42.682962 14986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:34485 every 8 connection(s)
I20260929 18:40:42.683822 14858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:42.685746 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14858
I20260929 18:40:42.685889 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:42.686192 11739 external_mini_cluster.cc:1428] Setting key 9bcc8bfdf91a14b80833d084ea436958
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:42.700618 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:42.685879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:59059 (local address 127.11.118.254:39065)
0929 18:40:42.686037 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:42.686041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:42.687089 (+ 1048us) server_negotiation.cc:407] Connection header received
0929 18:40:42.688624 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:42.688630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:42.688693 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:42.688786 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:42.690458 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.691022 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.692897 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.693125 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.693205 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:42.696392 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:42.696396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:42.696398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:42.696424 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:42.698011 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.698701 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.698704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.698706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.698790 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.699314 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.699320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.699325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.699589 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:42.699672 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:42.700375 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 18:40:42.700493 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:40:42.701591 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:42.686382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39065 (local address 127.11.118.193:59059)
0929 18:40:42.686927 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:42.686981 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:42.688401 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:42.688984 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:42.688991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:42.689536 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:42.690248 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:42.690309 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.691160 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.691167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:42.692741 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:42.692756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.693289 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.693296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:42.693547 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:42.694214 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:42.694236 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:42.696142 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:42.698139 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:42.698144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:42.698157 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:42.698556 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:42.699030 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:42.699036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:42.699041 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:42.699193 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:42.699699 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:42.699706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:42.700059 (+ 353us) client_negotiation.cc:769] Sending connection context
0929 18:40:42.700279 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 18:40:42.700672 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":170,"threads_started":1}
I20260929 18:40:42.702957 14987 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39065
I20260929 18:40:42.703227 14987 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:42.703786 14987 heartbeater.cc:511] Master 127.11.118.254:39065 requested a full tablet report, sending...
I20260929 18:40:42.705423 14796 ts_manager.cc:194] Registered new tserver with Master: 3e88ec72d4554cd39b539b40bc2075ef (127.11.118.193:34485)
I20260929 18:40:42.706630 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:59059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:42.717757 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:42.828784 14992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:42.829069 14992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:42.829140 14992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:42.833257 14992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:42.833352 14992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:42.833442 14992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:42.839039 14992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:42.840116 14992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:42.841150 14992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:42.851164 15000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:42.851259 14998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:42.851203 14992 server_base.cc:1034] running on GCE node
W20260929 18:40:42.851513 14997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:42.851908 14992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:42.852658 14992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:42.854038 14992 hybrid_clock.cc:648] HybridClock initialized: now 1790707242854013 us; error 47 us; skew 500 ppm
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:42.857064 14992 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:42.858088 14992 webserver.cc:460] Webserver started at http://127.11.118.194:43409/ using document root <none> and password file <none>
I20260929 18:40:42.858709 14992 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:42.858810 14992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:42.859054 14992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:42.861105 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "15ee4e49e0cb4d7994eba19525bad60c"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "d5cb94ac2b6f09db6c36f0858686d213"
server_key_iv: "f9bc728a9abd4702d483851860f0f5b5"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.861670 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "15ee4e49e0cb4d7994eba19525bad60c"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "d5cb94ac2b6f09db6c36f0858686d213"
server_key_iv: "f9bc728a9abd4702d483851860f0f5b5"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.865499 14992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:40:42.868137 15007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.869484 14992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:42.869614 14992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "15ee4e49e0cb4d7994eba19525bad60c"
format_stamp: "Formatted at 2026-09-29 18:40:42 on dist-test-slave-cgv7"
server_key: "d5cb94ac2b6f09db6c36f0858686d213"
server_key_iv: "f9bc728a9abd4702d483851860f0f5b5"
server_key_version: "encryptionkey@0"
I20260929 18:40:42.869728 14992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:42.893529 14992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:42.896708 14992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:42.896926 14992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:42.897527 14992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:42.898427 14992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:42.898495 14992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.898564 14992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:42.898617 14992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:42.907647 14992 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41173
I20260929 18:40:42.907668 15120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41173 every 8 connection(s)
I20260929 18:40:42.908583 14992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:42.914446 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 14992
I20260929 18:40:42.914583 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:42.914872 11739 external_mini_cluster.cc:1428] Setting key ffe1be86014523f1461cdaafacacf839
Sep 29 18:40:42 dist-test-slave-cgv7 krb5kdc[14751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707242, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:42.924377 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:42.910760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:50349 (local address 127.11.118.254:39065)
0929 18:40:42.910905 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:42.910909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:42.911941 (+ 1032us) server_negotiation.cc:407] Connection header received
0929 18:40:42.913511 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:42.913514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:42.913565 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:42.913642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:42.915153 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.915788 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.917396 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.917627 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.917714 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:42.920548 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:42.920552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:42.920554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:42.920589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:42.922200 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.922795 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.922798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.922800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.922887 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:42.923231 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:42.923235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:42.923237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:42.923445 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:42.923554 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:42.924072 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 18:40:42.924200 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:40:42.925262 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:42.911178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39065 (local address 127.11.118.194:50349)
0929 18:40:42.911736 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:42.911780 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:42.913256 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:42.913785 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:42.913791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:42.914230 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:42.914994 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:42.915015 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.915954 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.915957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:42.917262 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:42.917276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:42.917790 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:42.917797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:42.918047 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:42.918824 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:42.918842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:42.920345 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:42.922374 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:42.922383 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:42.922404 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:42.922685 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:42.922981 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:42.922985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:42.922987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:42.923133 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:42.923550 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:42.923557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:42.923821 (+ 264us) client_negotiation.cc:769] Sending connection context
0929 18:40:42.924032 (+ 211us) client_negotiation.cc:240] Negotiation successful
0929 18:40:42.924371 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":132,"threads_started":1}
I20260929 18:40:42.926618 15121 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39065
I20260929 18:40:42.926877 15121 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:42.927452 15121 heartbeater.cc:511] Master 127.11.118.254:39065 requested a full tablet report, sending...
I20260929 18:40:42.928618 14796 ts_manager.cc:194] Registered new tserver with Master: 15ee4e49e0cb4d7994eba19525bad60c (127.11.118.194:41173)
I20260929 18:40:42.929210 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:50349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:42.947268 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:43.058123 15126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:43.058485 15126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:43.058537 15126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:43.062582 15126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:43.062642 15126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:43.062703 15126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:43.068085 15126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:43.069142 15126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:43.070106 15126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:43.080248 15126 server_base.cc:1034] running on GCE node
W20260929 18:40:43.080125 15134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:43.080194 15131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:43.080125 15132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:43.080935 15126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:43.081625 15126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:43.082808 15126 hybrid_clock.cc:648] HybridClock initialized: now 1790707243082784 us; error 34 us; skew 500 ppm
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707243, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:43.085678 15126 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:43.086611 15126 webserver.cc:460] Webserver started at http://127.11.118.195:33657/ using document root <none> and password file <none>
I20260929 18:40:43.087143 15126 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:43.087203 15126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:43.087395 15126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:43.089489 15126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "e5b5a07a799d4ce6a6e7f10c06ef1384"
format_stamp: "Formatted at 2026-09-29 18:40:43 on dist-test-slave-cgv7"
server_key: "2931a94060b2a76cba1a71d9416810eb"
server_key_iv: "0128e5fe7255c3483e874bc9a6d1a9da"
server_key_version: "encryptionkey@0"
I20260929 18:40:43.090000 15126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "e5b5a07a799d4ce6a6e7f10c06ef1384"
format_stamp: "Formatted at 2026-09-29 18:40:43 on dist-test-slave-cgv7"
server_key: "2931a94060b2a76cba1a71d9416810eb"
server_key_iv: "0128e5fe7255c3483e874bc9a6d1a9da"
server_key_version: "encryptionkey@0"
I20260929 18:40:43.093765 15126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:40:43.096297 15141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:43.097559 15126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:43.097710 15126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "e5b5a07a799d4ce6a6e7f10c06ef1384"
format_stamp: "Formatted at 2026-09-29 18:40:43 on dist-test-slave-cgv7"
server_key: "2931a94060b2a76cba1a71d9416810eb"
server_key_iv: "0128e5fe7255c3483e874bc9a6d1a9da"
server_key_version: "encryptionkey@0"
I20260929 18:40:43.097831 15126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:43.115000 15126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:43.118144 15126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:43.118391 15126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:43.119001 15126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:43.119833 15126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:43.119904 15126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:43.119973 15126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:43.120025 15126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:43.129185 15126 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:33911
I20260929 18:40:43.129209 15254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:33911 every 8 connection(s)
I20260929 18:40:43.130183 15126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:43.134130 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15126
I20260929 18:40:43.134261 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:43.134608 11739 external_mini_cluster.cc:1428] Setting key 031b836a4a988d4690305bf36b423ac1
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[14751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707243, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:43.144747 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:43.132075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:44663 (local address 127.11.118.254:39065)
0929 18:40:43.132238 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:43.132243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:43.133098 (+ 855us) server_negotiation.cc:407] Connection header received
0929 18:40:43.134815 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:43.134818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:43.134877 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:43.135014 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:43.136420 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.137054 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.138212 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.138511 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.138624 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:43.141297 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:43.141300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:43.141303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:43.141328 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:43.142808 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:43.143303 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:43.143306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:43.143307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:43.143400 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:43.143693 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:43.143695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:43.143696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:43.143865 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:43.143947 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:43.144513 (+ 566us) server_negotiation.cc:299] Negotiation successful
0929 18:40:43.144624 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 18:40:43.145646 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:43.132473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39065 (local address 127.11.118.195:44663)
0929 18:40:43.132942 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:43.132986 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:43.134531 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:43.135222 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:43.135230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:43.135683 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:43.136273 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:43.136287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.137187 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.137195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:43.138084 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:43.138093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.138662 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.138666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:43.138873 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:43.139597 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:43.139613 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:43.141143 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:43.142934 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:43.142938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:43.142952 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:43.143209 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:43.143483 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:43.143486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:43.143487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:43.143615 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:43.143985 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:43.143990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:43.144230 (+ 240us) client_negotiation.cc:769] Sending connection context
0929 18:40:43.144418 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 18:40:43.144681 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260929 18:40:43.147032 15255 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39065
I20260929 18:40:43.147285 15255 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:43.147883 15255 heartbeater.cc:511] Master 127.11.118.254:39065 requested a full tablet report, sending...
I20260929 18:40:43.148861 14796 ts_manager.cc:194] Registered new tserver with Master: e5b5a07a799d4ce6a6e7f10c06ef1384 (127.11.118.195:33911)
I20260929 18:40:43.149441 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:44663
I20260929 18:40:43.160521 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707243, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[14751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707243, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:43.186718 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:43.177826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55648 (local address 127.11.118.254:39065)
0929 18:40:43.177968 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:43.177972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:43.178045 (+ 73us) server_negotiation.cc:407] Connection header received
0929 18:40:43.178168 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:43.178171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:43.178216 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:43.178380 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:43.179113 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.179646 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.180474 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:43.180694 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:43.180784 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:43.183387 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:43.183390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:43.183392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:43.183419 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:43.184960 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:43.185415 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:43.185418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:43.185419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:43.185501 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:43.185852 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:43.185855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:43.185860 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:43.186102 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:43.186211 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:43.186495 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 18:40:43.186595 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 18:40:43.187121 14796 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55648
I20260929 18:40:43.188042 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14858
I20260929 18:40:43.193608 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14992
I20260929 18:40:43.199119 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15126
I20260929 18:40:43.204363 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 14764
2026-09-29T18:40:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2201 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[15267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[15267](info): set up 2 sockets
Sep 29 18:40:43 dist-test-slave-cgv7 krb5kdc[15267](info): commencing operation
krb5kdc: starting...
W20260929 18:40:44.250496 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:44 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:44.385761 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38981
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:44.504995 15280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:44.505295 15280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:44.505338 15280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:44.509471 15280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:44.509554 15280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:44.509575 15280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:44.509593 15280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:44.509608 15280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:44.515226 15280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:44.516323 15280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:44.517318 15280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:44.526221 15286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:44.526301 15288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:44.526316 15285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:44.526895 15280 server_base.cc:1034] running on GCE node
I20260929 18:40:44.527856 15280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:44.529110 15280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:44.530365 15280 hybrid_clock.cc:648] HybridClock initialized: now 1790707244530351 us; error 32 us; skew 500 ppm
Sep 29 18:40:44 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:44.533564 15280 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:44.534683 15280 webserver.cc:460] Webserver started at http://127.11.118.254:43299/ using document root <none> and password file <none>
I20260929 18:40:44.535286 15280 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:44.535392 15280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:44.535642 15280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:44.537779 15280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "1476cbc746604ba591bdb25736938ac5"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "5fe2090095d8e2ada052dfcb74567788"
server_key_iv: "c9924fc1225acce8aa4756f005acb0b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.538451 15280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "1476cbc746604ba591bdb25736938ac5"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "5fe2090095d8e2ada052dfcb74567788"
server_key_iv: "c9924fc1225acce8aa4756f005acb0b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.542380 15280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:44.544934 15295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:44.546219 15280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:44.546415 15280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "1476cbc746604ba591bdb25736938ac5"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "5fe2090095d8e2ada052dfcb74567788"
server_key_iv: "c9924fc1225acce8aa4756f005acb0b0"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.546537 15280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:44.572330 15280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:44.575877 15280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:44.576097 15280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:44.583709 15280 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:38981
I20260929 18:40:44.583732 15347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:38981 every 8 connection(s)
I20260929 18:40:44.584741 15280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:44.587671 15348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:44.593145 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15280
I20260929 18:40:44.593254 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:44.593408 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: Bootstrap starting.
I20260929 18:40:44.593606 11739 external_mini_cluster.cc:1428] Setting key 75c8232abff2c8878a78f5e15e7c5da2
I20260929 18:40:44.596072 15348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:44.596855 15348 log.cc:824] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:44.598937 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: No bootstrap required, opened a new log
I20260929 18:40:44.602622 15348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } }
I20260929 18:40:44.602941 15348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:44.602993 15348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1476cbc746604ba591bdb25736938ac5, State: Initialized, Role: FOLLOWER
Sep 29 18:40:44 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:44.603506 15348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } }
I20260929 18:40:44.603684 15348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:44.603757 15348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:44.603868 15348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:44.605207 15348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } }
I20260929 18:40:44.605625 15348 leader_election.cc:302] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1476cbc746604ba591bdb25736938ac5; no voters:
I20260929 18:40:44.606015 15348 leader_election.cc:288] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:44.606178 15353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:44.606448 15353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [term 1 LEADER]: Becoming Leader. State: Replica: 1476cbc746604ba591bdb25736938ac5, State: Running, Role: LEADER
I20260929 18:40:44.606878 15353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } }
I20260929 18:40:44.607493 15348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:44.608718 15354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1476cbc746604ba591bdb25736938ac5. Latest consensus state: current_term: 1 leader_uuid: "1476cbc746604ba591bdb25736938ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } } }
I20260929 18:40:44.609162 15354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:44.609046 15355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1476cbc746604ba591bdb25736938ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1476cbc746604ba591bdb25736938ac5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38981 } } }
I20260929 18:40:44.609474 15355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:44.610209 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:44.595079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.11.118.254:38981)
0929 18:40:44.595528 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:44.595544 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:44.595582 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:40:44.596352 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:44.596373 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:44.596732 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:44.597105 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:44.598099 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.599262 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.600209 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.600692 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.600865 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:44.603707 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:44.603715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:44.603735 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:44.603776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:44.605968 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:44.606647 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:44.606653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:44.606659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:44.606807 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:44.607177 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:44.607180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:44.607182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:44.607749 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:44.607982 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:44.608783 (+ 801us) server_negotiation.cc:299] Negotiation successful
0929 18:40:44.609124 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":134,"threads_started":1}
W20260929 18:40:44.612833 15368 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:44.612968 15368 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:44.613063 15366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:44.613749 15366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:44.619163 15366 catalog_manager.cc:1347] Generated new cluster ID: 54476761566e40aca6883166196e454c
I20260929 18:40:44.619247 15366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:44.625578 15366 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:44.626611 15366 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:44.636583 15366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1476cbc746604ba591bdb25736938ac5: Generated new TSK 0
I20260929 18:40:44.637578 15366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:44.667603 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:44.780606 15374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:44.780885 15374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:44.780931 15374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:44.784993 15374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:44.785058 15374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:44.785161 15374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:44.790470 15374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:44.791642 15374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:44.792688 15374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:44.802337 15380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:44.802475 15374 server_base.cc:1034] running on GCE node
W20260929 18:40:44.802304 15379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:44.802320 15382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:44.802946 15374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:44.803625 15374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:44.804806 15374 hybrid_clock.cc:648] HybridClock initialized: now 1790707244804782 us; error 45 us; skew 500 ppm
Sep 29 18:40:44 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:44.807616 15374 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:44.808555 15374 webserver.cc:460] Webserver started at http://127.11.118.193:41031/ using document root <none> and password file <none>
I20260929 18:40:44.809064 15374 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:44.809122 15374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:44.809325 15374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:44.811388 15374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "19c6047ae9d249cb8f95a3c99e1a283b"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "a9696c631f74020e1907f9caa576788a"
server_key_iv: "c3b885cbf85384dd08dced9d99ea3531"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.811896 15374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "19c6047ae9d249cb8f95a3c99e1a283b"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "a9696c631f74020e1907f9caa576788a"
server_key_iv: "c3b885cbf85384dd08dced9d99ea3531"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.815582 15374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:40:44.817916 15389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:44.819190 15374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:44.819324 15374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "19c6047ae9d249cb8f95a3c99e1a283b"
format_stamp: "Formatted at 2026-09-29 18:40:44 on dist-test-slave-cgv7"
server_key: "a9696c631f74020e1907f9caa576788a"
server_key_iv: "c3b885cbf85384dd08dced9d99ea3531"
server_key_version: "encryptionkey@0"
I20260929 18:40:44.819448 15374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:44.852674 15374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:44.856172 15374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:44.856383 15374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:44.856993 15374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:44.857857 15374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:44.857928 15374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:44.857999 15374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:44.858045 15374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:44.866796 15374 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:33485
I20260929 18:40:44.866819 15502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:33485 every 8 connection(s)
I20260929 18:40:44.867703 15374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:44.875242 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15374
I20260929 18:40:44.875389 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:44.875691 11739 external_mini_cluster.cc:1428] Setting key 83434649355e2824332dd3e08f5c52a0
Sep 29 18:40:44 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:44.882638 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:44.869787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:48531 (local address 127.11.118.254:38981)
0929 18:40:44.869959 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:44.869963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:44.870830 (+ 867us) server_negotiation.cc:407] Connection header received
0929 18:40:44.872173 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:44.872176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:44.872223 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:44.872306 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:44.873517 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.874030 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.875411 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.875684 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.875819 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:44.878897 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:44.878900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:44.878902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:44.878928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:44.880427 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:44.881020 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:44.881025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:44.881026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:44.881101 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:44.881472 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:44.881475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:44.881477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:44.881672 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:44.881781 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:44.882339 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 18:40:44.882471 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 18:40:44.883507 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:44.870189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38981 (local address 127.11.118.193:48531)
0929 18:40:44.870661 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:44.870705 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:44.871946 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:44.872491 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:44.872497 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:44.872886 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:44.873383 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:44.873397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.874130 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.874132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:44.875245 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:44.875258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:44.875872 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:44.875877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:44.876082 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:44.876779 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:44.876797 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:44.878695 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:44.880566 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:44.880573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:44.880592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:44.880899 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:44.881206 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:44.881209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:44.881212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:44.881369 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:44.881781 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:44.881789 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:44.882066 (+ 277us) client_negotiation.cc:769] Sending connection context
0929 18:40:44.882262 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 18:40:44.882654 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":111,"threads_started":1}
I20260929 18:40:44.884840 15503 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38981
I20260929 18:40:44.885106 15503 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:44.885658 15503 heartbeater.cc:511] Master 127.11.118.254:38981 requested a full tablet report, sending...
I20260929 18:40:44.887305 15312 ts_manager.cc:194] Registered new tserver with Master: 19c6047ae9d249cb8f95a3c99e1a283b (127.11.118.193:33485)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:44.888679 15312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:48531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:44.907019 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:45.019114 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:45.019399 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:45.019444 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:45.023710 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:45.023774 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:45.023885 15508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:45.029474 15508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:45.030575 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:45.031546 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:45.041496 15514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:45.041580 15508 server_base.cc:1034] running on GCE node
W20260929 18:40:45.041580 15513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:45.041514 15516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:45.042337 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:45.043053 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:45.044240 15508 hybrid_clock.cc:648] HybridClock initialized: now 1790707245044214 us; error 37 us; skew 500 ppm
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:45.047075 15508 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:45.047959 15508 webserver.cc:460] Webserver started at http://127.11.118.194:41679/ using document root <none> and password file <none>
I20260929 18:40:45.048527 15508 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:45.048619 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:45.048851 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:45.051278 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "75cdb98fe5064ebf98a738e450633243"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "9cc63f87d11ad18e0bf7c86fcb4f1edf"
server_key_iv: "07f17d76c544033caa0596b21e480ab4"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.051878 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "75cdb98fe5064ebf98a738e450633243"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "9cc63f87d11ad18e0bf7c86fcb4f1edf"
server_key_iv: "07f17d76c544033caa0596b21e480ab4"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.055744 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:45.058387 15523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.059687 15508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:45.059826 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "75cdb98fe5064ebf98a738e450633243"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "9cc63f87d11ad18e0bf7c86fcb4f1edf"
server_key_iv: "07f17d76c544033caa0596b21e480ab4"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.059950 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:45.073900 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:45.077435 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:45.077649 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:45.078306 15508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:45.079216 15508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:45.079264 15508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.079304 15508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:45.079319 15508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.088706 15508 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:43763
I20260929 18:40:45.088726 15636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:43763 every 8 connection(s)
I20260929 18:40:45.089708 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:45.094030 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15508
I20260929 18:40:45.094184 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:45.094589 11739 external_mini_cluster.cc:1428] Setting key b6ec15adfb30fba421dde245e16534f5
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:45.105882 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.091669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:48591 (local address 127.11.118.254:38981)
0929 18:40:45.091819 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:45.091824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:45.092794 (+ 970us) server_negotiation.cc:407] Connection header received
0929 18:40:45.094436 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:45.094441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:45.094505 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:45.094596 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:45.095994 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.096748 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.098477 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.098689 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.098773 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.101644 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:45.101650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:45.101656 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:45.101690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:45.103551 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.104136 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.104141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.104145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.104237 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.104563 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.104570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.104573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.104851 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:45.104959 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:45.105518 (+ 559us) server_negotiation.cc:299] Negotiation successful
0929 18:40:45.105629 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:40:45.106858 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.092126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38981 (local address 127.11.118.194:48591)
0929 18:40:45.092621 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:45.092665 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:45.094123 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:45.094794 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:45.094808 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:45.095310 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:45.095830 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:45.095844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.096925 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.096929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:45.098343 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:45.098359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.098861 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.098868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:45.099086 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.099729 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:45.099746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:45.101448 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:45.103688 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:45.103693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:45.103706 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:45.104020 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:45.104329 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:45.104331 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:45.104333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:45.104468 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:45.104985 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:45.104991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:45.105251 (+ 260us) client_negotiation.cc:769] Sending connection context
0929 18:40:45.105441 (+ 190us) client_negotiation.cc:240] Negotiation successful
0929 18:40:45.105755 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:45.108439 15637 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38981
I20260929 18:40:45.108723 15637 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:45.109416 15637 heartbeater.cc:511] Master 127.11.118.254:38981 requested a full tablet report, sending...
I20260929 18:40:45.110669 15312 ts_manager.cc:194] Registered new tserver with Master: 75cdb98fe5064ebf98a738e450633243 (127.11.118.194:43763)
I20260929 18:40:45.111590 15312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:48591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:45.125234 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:45.237751 15642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:45.238029 15642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:45.238075 15642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:45.242765 15642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:45.242837 15642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:45.242908 15642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:45.248560 15642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:45.249725 15642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:45.250840 15642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:45.261157 15650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:45.261168 15647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:45.261184 15648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:45.261480 15642 server_base.cc:1034] running on GCE node
I20260929 18:40:45.261936 15642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:45.262681 15642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:45.263846 15642 hybrid_clock.cc:648] HybridClock initialized: now 1790707245263814 us; error 56 us; skew 500 ppm
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:45.266860 15642 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:45.267763 15642 webserver.cc:460] Webserver started at http://127.11.118.195:43219/ using document root <none> and password file <none>
I20260929 18:40:45.268361 15642 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:45.268455 15642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:45.268687 15642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:45.270895 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "dfba067d4f964a8ebb8d8ecb44605166"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "32a2824d2a89ca5440ca6c80bff1afb7"
server_key_iv: "64acfe0f3e1e342e6a5dfc383a728e91"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.271520 15642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "dfba067d4f964a8ebb8d8ecb44605166"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "32a2824d2a89ca5440ca6c80bff1afb7"
server_key_iv: "64acfe0f3e1e342e6a5dfc383a728e91"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.275578 15642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 18:40:45.278200 15657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.279664 15642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:40:45.279837 15642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "dfba067d4f964a8ebb8d8ecb44605166"
format_stamp: "Formatted at 2026-09-29 18:40:45 on dist-test-slave-cgv7"
server_key: "32a2824d2a89ca5440ca6c80bff1afb7"
server_key_iv: "64acfe0f3e1e342e6a5dfc383a728e91"
server_key_version: "encryptionkey@0"
I20260929 18:40:45.279963 15642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:45.303846 15642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:45.307261 15642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:45.307503 15642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:45.308117 15642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:45.309053 15642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:45.309123 15642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.309190 15642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:45.309239 15642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:45.318205 15642 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:35759
I20260929 18:40:45.318222 15770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:35759 every 8 connection(s)
I20260929 18:40:45.319114 15642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:45.322088 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15642
I20260929 18:40:45.322221 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:45.322616 11739 external_mini_cluster.cc:1428] Setting key 1888a86700a3e07e6ae046aa95db859d
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:45.333446 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.320948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:36657 (local address 127.11.118.254:38981)
0929 18:40:45.321108 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:45.321113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:45.321985 (+ 872us) server_negotiation.cc:407] Connection header received
0929 18:40:45.323959 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:45.323965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:45.324019 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:45.324122 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:45.325466 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.326053 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.327237 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.327477 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.327588 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.330093 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:45.330097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:45.330099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:45.330134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:45.331403 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.331933 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.331937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.331938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.332010 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.332345 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.332350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.332355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.332537 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:45.332641 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:45.333151 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 18:40:45.333270 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 18:40:45.334360 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.321262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38981 (local address 127.11.118.195:36657)
0929 18:40:45.321742 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:45.321850 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:45.323682 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:45.324315 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:45.324323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:45.324720 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:45.325302 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:45.325317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.326181 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.326186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:45.327106 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:45.327115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.327638 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.327642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:45.327816 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.328371 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:45.328387 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:45.329905 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:45.331521 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:45.331527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:45.331543 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:45.331797 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:45.332104 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:45.332108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:45.332110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:45.332249 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:45.332651 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:45.332659 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:45.332920 (+ 261us) client_negotiation.cc:769] Sending connection context
0929 18:40:45.333114 (+ 194us) client_negotiation.cc:240] Negotiation successful
0929 18:40:45.333503 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":111,"threads_started":1}
I20260929 18:40:45.335580 15771 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38981
I20260929 18:40:45.335911 15771 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:45.336509 15771 heartbeater.cc:511] Master 127.11.118.254:38981 requested a full tablet report, sending...
I20260929 18:40:45.337514 15312 ts_manager.cc:194] Registered new tserver with Master: dfba067d4f964a8ebb8d8ecb44605166 (127.11.118.195:35759)
I20260929 18:40:45.338084 15312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:36657
I20260929 18:40:45.348654 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:45.375361 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.365374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.11.118.193:33485)
0929 18:40:45.365685 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:45.365699 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:45.365721 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:40:45.365790 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:45.365796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:45.366014 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:45.366138 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:45.366925 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.367749 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.368612 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.369032 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.369120 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.371691 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:45.371700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:45.371713 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:45.371738 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:45.373514 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.373942 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.373948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.373949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.374027 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.374350 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.374353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.374354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.374627 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:45.374804 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:45.374978 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 18:40:45.375141 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":109,"threads_started":1}
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:45.386741 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:45.378381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.11.118.254:38981)
0929 18:40:45.378527 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:45.378533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:45.378548 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:45.378705 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:45.378707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:45.378748 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:45.378833 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:45.379442 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.379986 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.380770 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:45.380991 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:45.381100 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:45.383506 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:45.383510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:45.383512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:45.383536 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:45.385119 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.385510 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.385512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.385513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.385586 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:45.385849 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:45.385852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:45.385853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:45.386062 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:45.386159 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:45.386453 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 18:40:45.386551 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 18:40:45.387147 15312 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55670
I20260929 18:40:45.387866 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15374
I20260929 18:40:45.393539 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15508
I20260929 18:40:45.398921 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15642
I20260929 18:40:45.403962 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15280
2026-09-29T18:40:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2200 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15783](info): set up 2 sockets
Sep 29 18:40:45 dist-test-slave-cgv7 krb5kdc[15783](info): commencing operation
krb5kdc: starting...
W20260929 18:40:46.454555 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:46 dist-test-slave-cgv7 krb5kdc[15783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:46.585609 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37683
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:37683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:46.698944 15796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:46.699290 15796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:46.699365 15796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:46.703987 15796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:46.704101 15796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:46.704139 15796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:46.704181 15796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:46.704221 15796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:46.709898 15796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:37683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:46.711520 15796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:46.712641 15796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:46.721467 15804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:46.721480 15801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:46.721663 15796 server_base.cc:1034] running on GCE node
W20260929 18:40:46.721480 15802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:46.722481 15796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:46.723613 15796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:46.724785 15796 hybrid_clock.cc:648] HybridClock initialized: now 1790707246724768 us; error 34 us; skew 500 ppm
Sep 29 18:40:46 dist-test-slave-cgv7 krb5kdc[15783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707246, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:46.727645 15796 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:46.728351 15796 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:46.728416 15796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:46.728585 15796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:46.730683 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "9f55fd03f1c6404e9a211e3158e8add7"
format_stamp: "Formatted at 2026-09-29 18:40:46 on dist-test-slave-cgv7"
server_key: "8f5af53a41a293263637e8150186e62d"
server_key_iv: "d8706a3f15d3b1c701679e626332e527"
server_key_version: "encryptionkey@0"
I20260929 18:40:46.731191 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "9f55fd03f1c6404e9a211e3158e8add7"
format_stamp: "Formatted at 2026-09-29 18:40:46 on dist-test-slave-cgv7"
server_key: "8f5af53a41a293263637e8150186e62d"
server_key_iv: "d8706a3f15d3b1c701679e626332e527"
server_key_version: "encryptionkey@0"
I20260929 18:40:46.734872 15796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:40:46.737353 15810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:46.738775 15796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:46.738965 15796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "9f55fd03f1c6404e9a211e3158e8add7"
format_stamp: "Formatted at 2026-09-29 18:40:46 on dist-test-slave-cgv7"
server_key: "8f5af53a41a293263637e8150186e62d"
server_key_iv: "d8706a3f15d3b1c701679e626332e527"
server_key_version: "encryptionkey@0"
I20260929 18:40:46.739071 15796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:46.755975 15796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:46.759450 15796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:46.759676 15796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:46.766593 15796 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:37683
I20260929 18:40:46.766894 15862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:37683 every 8 connection(s)
I20260929 18:40:46.767334 15796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:46.770071 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:46.772322 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15796
I20260929 18:40:46.772487 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:46.772768 11739 external_mini_cluster.cc:1428] Setting key a570df106b88b90c1c1dc23f2baccc07
I20260929 18:40:46.775996 15863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: Bootstrap starting.
I20260929 18:40:46.778210 15863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:46.778985 15863 log.cc:824] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:46.780936 15863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: No bootstrap required, opened a new log
Sep 29 18:40:46 dist-test-slave-cgv7 krb5kdc[15783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:46.784602 15863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } }
I20260929 18:40:46.784925 15863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:46.785012 15863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f55fd03f1c6404e9a211e3158e8add7, State: Initialized, Role: FOLLOWER
I20260929 18:40:46.785547 15863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } }
I20260929 18:40:46.785687 15863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:46.785784 15863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:46.785915 15863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:46.787093 15863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } }
I20260929 18:40:46.787465 15863 leader_election.cc:302] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f55fd03f1c6404e9a211e3158e8add7; no voters:
I20260929 18:40:46.787722 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:46.774103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.11.118.254:37683)
0929 18:40:46.774541 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:46.774557 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:46.774591 (+ 34us) server_negotiation.cc:407] Connection header received
0929 18:40:46.775211 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:46.775227 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:46.775550 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:46.775891 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:46.776718 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:46.777683 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:46.778648 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:46.779158 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:46.779331 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:46.782016 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:46.782021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:46.782035 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:46.782060 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:46.784372 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:46.784935 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:46.784944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:46.784952 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:46.785135 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:46.785437 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:46.785439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:46.785440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:46.786004 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:46.786178 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:46.786512 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 18:40:46.786826 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":150,"threads_started":1}
I20260929 18:40:46.787853 15863 leader_election.cc:288] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:46.787932 15868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:46.788172 15868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [term 1 LEADER]: Becoming Leader. State: Replica: 9f55fd03f1c6404e9a211e3158e8add7, State: Running, Role: LEADER
I20260929 18:40:46.788544 15868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } }
I20260929 18:40:46.789218 15863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:46.790210 15869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f55fd03f1c6404e9a211e3158e8add7. Latest consensus state: current_term: 1 leader_uuid: "9f55fd03f1c6404e9a211e3158e8add7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } } }
I20260929 18:40:46.790462 15869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:46.790534 15870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f55fd03f1c6404e9a211e3158e8add7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f55fd03f1c6404e9a211e3158e8add7" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37683 } } }
I20260929 18:40:46.790704 15870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7 [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:46.794181 15884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:46.794252 15884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:46.794361 15877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:46.795188 15877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:46.801496 15877 catalog_manager.cc:1347] Generated new cluster ID: 85be723ee9814f4a8f578c1eb2071c20
I20260929 18:40:46.801587 15877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:46.822686 15877 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:46.823768 15877 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:46.828687 15877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: Generated new TSK 0
I20260929 18:40:46.829385 15877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:46.861721 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:46.978138 15889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:46.978531 15889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:46.978624 15889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:46.982805 15889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:46.982933 15889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:46.983039 15889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:46.988718 15889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:46.989944 15889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:46.991125 15889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:47.001303 15894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.001292 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.001292 15895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.001722 15889 server_base.cc:1034] running on GCE node
I20260929 18:40:47.002219 15889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:47.003001 15889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:47.004190 15889 hybrid_clock.cc:648] HybridClock initialized: now 1790707247004181 us; error 44 us; skew 500 ppm
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:47.007355 15889 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:47.008481 15889 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:47.008592 15889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:47.009001 15889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:47.011474 15889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "07886a4493a54f7c8d3169adb2a2fbf5"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "0f9f40f50667e9005de08437e41def85"
server_key_iv: "ee534c852de5c21a39c9ece5d37f1766"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.012120 15889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "07886a4493a54f7c8d3169adb2a2fbf5"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "0f9f40f50667e9005de08437e41def85"
server_key_iv: "ee534c852de5c21a39c9ece5d37f1766"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.016294 15889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 18:40:47.019059 15903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.020375 15889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:47.020526 15889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "07886a4493a54f7c8d3169adb2a2fbf5"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "0f9f40f50667e9005de08437e41def85"
server_key_iv: "ee534c852de5c21a39c9ece5d37f1766"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.020653 15889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:47.036404 15889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:47.040105 15889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:47.040333 15889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:47.040941 15889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:47.041792 15889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:47.041867 15889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.041939 15889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:47.042018 15889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.052007 15889 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:46281
I20260929 18:40:47.052021 16016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:46281 every 8 connection(s)
I20260929 18:40:47.052609 15889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:47.059115 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 15889
I20260929 18:40:47.059250 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:47.059578 11739 external_mini_cluster.cc:1428] Setting key 25b56adf2c4dc32a77caae1dce37c5af
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:47.069322 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.054395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36409 (local address 127.11.118.254:37683)
0929 18:40:47.054567 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.054571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.055787 (+ 1216us) server_negotiation.cc:407] Connection header received
0929 18:40:47.056720 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.056722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.056772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.056883 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:47.058249 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.058848 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.060432 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.060671 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.060846 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.064807 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:47.064815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:47.064822 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:47.064866 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:47.066879 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.067531 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.067534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.067535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.067621 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.068074 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.068078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.068079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.068293 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:47.068411 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.068978 (+ 567us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.069128 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:47.070259 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.054766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37683 (local address 127.11.118.193:36409)
0929 18:40:47.055621 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.055663 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.056511 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.057159 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.057167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.057548 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:47.058087 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.058101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.058993 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.058999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.060298 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.060308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.060832 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.060837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.061039 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.061733 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:47.061760 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:47.064486 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:47.067045 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.067055 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.067076 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.067388 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.067779 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.067782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.067783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.067940 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.068407 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:47.068416 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:47.068702 (+ 286us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.068921 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.069291 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":110,"threads_started":1}
I20260929 18:40:47.071818 16017 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37683
I20260929 18:40:47.072335 16017 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:47.073009 16017 heartbeater.cc:511] Master 127.11.118.254:37683 requested a full tablet report, sending...
I20260929 18:40:47.074941 15827 ts_manager.cc:194] Registered new tserver with Master: 07886a4493a54f7c8d3169adb2a2fbf5 (127.11.118.193:46281)
I20260929 18:40:47.076278 15827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:36409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:47.090934 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:47.204327 16022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:47.204643 16022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:47.204730 16022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:47.208940 16022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:47.209038 16022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:47.209136 16022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:47.214776 16022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:47.215925 16022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:47.216977 16022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:47.226627 16027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.226641 16028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.226646 16030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.227072 16022 server_base.cc:1034] running on GCE node
I20260929 18:40:47.227571 16022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:47.228219 16022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:47.229426 16022 hybrid_clock.cc:648] HybridClock initialized: now 1790707247229394 us; error 43 us; skew 500 ppm
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:47.232455 16022 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:47.233217 16022 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:47.233290 16022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:47.233457 16022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:47.235599 16022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "46cefda75cd24bcdb7c994db3f5c3be1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "b6a10342e2d8837ada5c9da727193111"
server_key_iv: "f1b0061d08bb81945245b30b0a20d379"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.236147 16022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "46cefda75cd24bcdb7c994db3f5c3be1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "b6a10342e2d8837ada5c9da727193111"
server_key_iv: "f1b0061d08bb81945245b30b0a20d379"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.240106 16022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 18:40:47.242741 16036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.244213 16022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:47.244365 16022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "46cefda75cd24bcdb7c994db3f5c3be1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "b6a10342e2d8837ada5c9da727193111"
server_key_iv: "f1b0061d08bb81945245b30b0a20d379"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.244473 16022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:47.258080 16022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:47.261636 16022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:47.261857 16022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:47.262543 16022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:47.263357 16022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:47.263402 16022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.263442 16022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:47.263458 16022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.272210 16022 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:37317
I20260929 18:40:47.272229 16149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:37317 every 8 connection(s)
I20260929 18:40:47.272890 16022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:47.277650 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16022
I20260929 18:40:47.277792 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:47.278124 11739 external_mini_cluster.cc:1428] Setting key 9c8b2968c8f2a950f076b78d0d331b3b
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:47.288726 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.274874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38569 (local address 127.11.118.254:37683)
0929 18:40:47.275019 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.275023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.276197 (+ 1174us) server_negotiation.cc:407] Connection header received
0929 18:40:47.277049 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.277052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.277098 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.277174 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:47.278996 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.279762 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.281577 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.281972 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.282095 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.284658 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:47.284661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:47.284663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:47.284687 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:47.286365 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.287005 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.287009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.287011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.287105 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.287508 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.287514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.287519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.287794 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:47.287910 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.288481 (+ 571us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.288599 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:40:47.289659 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.275275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37683 (local address 127.11.118.194:38569)
0929 18:40:47.276048 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.276088 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.276853 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.277498 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.277509 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.278081 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:47.278787 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.278808 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.279928 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.279932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.281428 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.281443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.282144 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.282147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.282382 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.283007 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:47.283024 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:47.284475 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:47.286508 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.286514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.286532 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.286872 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.287234 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.287237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.287238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.287389 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.287907 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:47.287912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:47.288242 (+ 330us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.288415 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.288691 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260929 18:40:47.291092 16150 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37683
I20260929 18:40:47.291401 16150 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:47.291977 16150 heartbeater.cc:511] Master 127.11.118.254:37683 requested a full tablet report, sending...
I20260929 18:40:47.293109 15827 ts_manager.cc:194] Registered new tserver with Master: 46cefda75cd24bcdb7c994db3f5c3be1 (127.11.118.194:37317)
I20260929 18:40:47.293701 15827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:38569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:47.310139 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:47.421305 16155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:47.421630 16155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:47.421712 16155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:47.426247 16155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:47.426373 16155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:47.426471 16155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:47.432878 16155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:47.434128 16155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:47.435240 16155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:47.445195 16161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.445228 16163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.445360 16155 server_base.cc:1034] running on GCE node
W20260929 18:40:47.445214 16160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.445950 16155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:47.446686 16155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:47.447845 16155 hybrid_clock.cc:648] HybridClock initialized: now 1790707247447826 us; error 33 us; skew 500 ppm
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:47.450865 16155 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:47.451661 16155 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:47.451727 16155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:47.451951 16155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:47.454123 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "31346cf196174ddbb2477a9a0aaf83b1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "88e17189d4cc00547b6ac297ef9fa6f5"
server_key_iv: "8811e462ade477e16f32d9759c095591"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.454795 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "31346cf196174ddbb2477a9a0aaf83b1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "88e17189d4cc00547b6ac297ef9fa6f5"
server_key_iv: "8811e462ade477e16f32d9759c095591"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.458706 16155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 18:40:47.461241 16169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.462551 16155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:47.462684 16155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "31346cf196174ddbb2477a9a0aaf83b1"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "88e17189d4cc00547b6ac297ef9fa6f5"
server_key_iv: "8811e462ade477e16f32d9759c095591"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.462807 16155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:47.487339 16155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:47.490824 16155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:47.491029 16155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:47.491627 16155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:47.492507 16155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:47.492579 16155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.492652 16155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:47.492697 16155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.501471 16155 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:43063
I20260929 18:40:47.501490 16282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:43063 every 8 connection(s)
I20260929 18:40:47.502086 16155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:47.506896 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16155
I20260929 18:40:47.507010 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:47.507319 11739 external_mini_cluster.cc:1428] Setting key a2cb5ba3fee62a7e5140e8bdc5b58cdf
Sep 29 18:40:47 dist-test-slave-cgv7 krb5kdc[15783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707247, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:47.516112 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.503983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:46137 (local address 127.11.118.254:37683)
0929 18:40:47.504159 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.504164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.505318 (+ 1154us) server_negotiation.cc:407] Connection header received
0929 18:40:47.506186 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.506189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.506238 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.506341 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:47.507692 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.508439 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.509846 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.510081 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.510195 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.512741 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:47.512744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:47.512746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:47.512768 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:47.514164 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.514699 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.514702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.514703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.514779 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.515122 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.515124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.515125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.515295 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:47.515370 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.515889 (+ 519us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.515991 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 18:40:47.516993 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.504363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37683 (local address 127.11.118.195:46137)
0929 18:40:47.505160 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.505203 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.505981 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.506513 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.506519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.506916 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:47.507533 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.507549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.508565 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.508569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.509722 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.509732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.510232 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.510237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.510486 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.511098 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:47.511114 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:47.512579 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:47.514330 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.514335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.514349 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.514607 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.514885 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:47.514888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:47.514889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:47.515034 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:47.515393 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:47.515398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:47.515637 (+ 239us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.515847 (+ 210us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.516113 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
I20260929 18:40:47.518303 16283 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37683
I20260929 18:40:47.518631 16283 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:47.519105 16283 heartbeater.cc:511] Master 127.11.118.254:37683 requested a full tablet report, sending...
I20260929 18:40:47.520124 15827 ts_manager.cc:194] Registered new tserver with Master: 31346cf196174ddbb2477a9a0aaf83b1 (127.11.118.195:43063)
I20260929 18:40:47.520798 15827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:46137
I20260929 18:40:47.521518 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:40:47.531423 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.523679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.11.118.254:37683)
0929 18:40:47.523829 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.523834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.524030 (+ 196us) server_negotiation.cc:407] Connection header received
0929 18:40:47.524126 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.524128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.524166 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.524234 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:47.525029 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.525653 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.526547 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.526766 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.526850 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.528309 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:47.528312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:47.528314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:47.528336 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:47.529662 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.530093 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.530096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.530097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.530172 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:47.530486 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:47.530488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:47.530489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:47.530702 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:47.530808 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.531066 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.531214 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:40:47.534595 15825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:47.537513 15825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:47.552021 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.545955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:43063 (local address 127.0.0.1:33498)
0929 18:40:47.546330 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.546375 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.546505 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.547577 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.547583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.547606 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:47.547768 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.547775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.548942 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.548949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.550458 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.550469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.551425 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.551428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.551543 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.551578 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.551728 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.551852 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260929 18:40:47.552026 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.546077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.11.118.195:43063)
0929 18:40:47.547120 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.547133 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.547158 (+ 25us) server_negotiation.cc:407] Connection header received
0929 18:40:47.547238 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.547243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.547419 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.547589 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:47.547910 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.548808 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.550593 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.551292 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.551447 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.551586 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.551757 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.551852 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":247,"threads_started":1}
I20260929 18:40:47.552223 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.546710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:37317 (local address 127.0.0.1:38174)
0929 18:40:47.547152 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.547164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.547258 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.547958 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.547962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.547986 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:47.548104 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.548110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.549624 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.549628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.550723 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.550730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.551818 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.551823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.551918 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.551940 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.551991 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.552063 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":2,"spinlock_wait_cycles":8704,"thread_start_us":102,"threads_started":1}
I20260929 18:40:47.552400 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.545965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:46281 (local address 127.0.0.1:46756)
0929 18:40:47.547146 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.547161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.547258 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.547986 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.547989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.548007 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:47.548141 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.548150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.549716 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.549721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.550861 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.550871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.552042 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.552046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.552120 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.552140 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.552205 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.552273 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"mutex_wait_us":4,"spinlock_wait_cycles":12928,"thread_start_us":119,"threads_started":1}
I20260929 18:40:47.552836 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.546841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.11.118.194:37317)
0929 18:40:47.547485 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.547496 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.547518 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:40:47.547593 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.547597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.547779 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.547978 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:47.548215 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.549489 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.550840 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.551665 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.552423 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.552556 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.552599 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.552673 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":115,"threads_started":1}
I20260929 18:40:47.552845 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.546772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.11.118.193:46281)
0929 18:40:47.547485 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.547503 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.547526 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:40:47.547593 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.547597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.547779 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.548296 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:47.548453 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.549581 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.551001 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.551741 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.551862 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.551987 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.552548 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.552646 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":89,"threads_started":1}
I20260929 18:40:47.555599 15951 tablet_service.cc:1467] Processing CreateTablet for tablet 547cb39c589a4584b20ea633733f5781 (DEFAULT_TABLE table=test-table [id=e26ba8d1107646329f506467ed9b61fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:47.555684 16217 tablet_service.cc:1467] Processing CreateTablet for tablet 547cb39c589a4584b20ea633733f5781 (DEFAULT_TABLE table=test-table [id=e26ba8d1107646329f506467ed9b61fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:47.555684 16084 tablet_service.cc:1467] Processing CreateTablet for tablet 547cb39c589a4584b20ea633733f5781 (DEFAULT_TABLE table=test-table [id=e26ba8d1107646329f506467ed9b61fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:47.556640 15951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547cb39c589a4584b20ea633733f5781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:47.556682 16084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547cb39c589a4584b20ea633733f5781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:47.556690 16217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547cb39c589a4584b20ea633733f5781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:47.563436 16300 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: Bootstrap starting.
I20260929 18:40:47.564767 16301 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Bootstrap starting.
I20260929 18:40:47.564790 16302 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Bootstrap starting.
I20260929 18:40:47.565706 16300 tablet_bootstrap.cc:654] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:47.566884 16300 log.cc:824] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:47.566943 16301 tablet_bootstrap.cc:654] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:47.566943 16302 tablet_bootstrap.cc:654] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:47.568094 16301 log.cc:824] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:47.568095 16302 log.cc:824] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:47.569044 16300 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: No bootstrap required, opened a new log
I20260929 18:40:47.569233 16300 ts_tablet_manager.cc:1397] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:40:47.570030 16301 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: No bootstrap required, opened a new log
I20260929 18:40:47.570170 16302 tablet_bootstrap.cc:492] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: No bootstrap required, opened a new log
I20260929 18:40:47.570202 16301 ts_tablet_manager.cc:1397] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:40:47.570381 16302 ts_tablet_manager.cc:1397] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:40:47.573127 16302 raft_consensus.cc:348] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.573143 16300 raft_consensus.cc:348] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.573386 16302 raft_consensus.cc:374] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:47.573395 16300 raft_consensus.cc:374] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:47.573442 16300 raft_consensus.cc:729] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07886a4493a54f7c8d3169adb2a2fbf5, State: Initialized, Role: FOLLOWER
I20260929 18:40:47.573447 16302 raft_consensus.cc:729] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46cefda75cd24bcdb7c994db3f5c3be1, State: Initialized, Role: FOLLOWER
I20260929 18:40:47.573879 16302 consensus_queue.cc:260] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.573879 16300 consensus_queue.cc:260] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.573800 16301 raft_consensus.cc:348] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.574033 16301 raft_consensus.cc:374] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:47.574085 16301 raft_consensus.cc:729] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31346cf196174ddbb2477a9a0aaf83b1, State: Initialized, Role: FOLLOWER
I20260929 18:40:47.574486 16301 consensus_queue.cc:260] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.574548 16017 heartbeater.cc:503] Master 127.11.118.254:37683 was elected leader, sending a full tablet report...
I20260929 18:40:47.574892 16300 ts_tablet_manager.cc:1428] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:40:47.575191 16302 ts_tablet_manager.cc:1428] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:40:47.575275 16301 ts_tablet_manager.cc:1428] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260929 18:40:47.575529 16150 heartbeater.cc:503] Master 127.11.118.254:37683 was elected leader, sending a full tablet report...
I20260929 18:40:47.575610 16283 heartbeater.cc:503] Master 127.11.118.254:37683 was elected leader, sending a full tablet report...
I20260929 18:40:47.668936 16306 raft_consensus.cc:482] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:47.669202 16306 raft_consensus.cc:504] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.670857 16306 leader_election.cc:288] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31346cf196174ddbb2477a9a0aaf83b1 (127.11.118.195:43063), 46cefda75cd24bcdb7c994db3f5c3be1 (127.11.118.194:37317)
I20260929 18:40:47.676292 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.671178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:33359 (local address 127.11.118.195:43063)
0929 18:40:47.671372 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.671377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.671979 (+ 602us) server_negotiation.cc:407] Connection header received
0929 18:40:47.672118 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.672121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.672165 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.672262 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:47.672541 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.673349 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.675039 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.675698 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.675826 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.675881 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.676094 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.676165 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 18:40:47.676321 16310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.671107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:43063 (local address 127.11.118.193:33359)
0929 18:40:47.671867 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.671882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.672023 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.672271 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.672274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.672289 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:47.672419 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.672427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.673496 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.673500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.674884 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.674895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.675847 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.675851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.675951 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.675986 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.676069 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.676148 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":134,"threads_started":1}
I20260929 18:40:47.677546 16237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547cb39c589a4584b20ea633733f5781" candidate_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31346cf196174ddbb2477a9a0aaf83b1" is_pre_election: true
I20260929 18:40:47.677979 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.671549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:37317 (local address 127.11.118.193:56587)
0929 18:40:47.672816 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.672829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:47.672949 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:47.673268 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:47.673271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:47.673288 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:47.673460 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.673470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.675164 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.675167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.676283 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:47.676292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.677497 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.677503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:47.677628 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.677650 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:40:47.677743 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 18:40:47.677822 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":96,"threads_started":1}
I20260929 18:40:47.678025 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.671699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:56587 (local address 127.11.118.194:37317)
0929 18:40:47.671824 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.671829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.672926 (+ 1097us) server_negotiation.cc:407] Connection header received
0929 18:40:47.673045 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.673048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.673130 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.673242 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:47.673619 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.674380 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.676431 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.677051 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.677145 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.677351 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.677822 (+ 471us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.677909 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:40:47.678440 16237 raft_consensus.cc:2454] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07886a4493a54f7c8d3169adb2a2fbf5 in term 0.
I20260929 18:40:47.678750 16104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547cb39c589a4584b20ea633733f5781" candidate_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" is_pre_election: true
I20260929 18:40:47.678977 15905 leader_election.cc:302] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07886a4493a54f7c8d3169adb2a2fbf5, 31346cf196174ddbb2477a9a0aaf83b1; no voters:
I20260929 18:40:47.679041 16104 raft_consensus.cc:2454] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07886a4493a54f7c8d3169adb2a2fbf5 in term 0.
I20260929 18:40:47.679256 16306 raft_consensus.cc:2781] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:47.679355 16306 raft_consensus.cc:482] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:47.679419 16306 raft_consensus.cc:3037] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:47.680495 16306 raft_consensus.cc:504] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.680852 16306 leader_election.cc:288] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [CANDIDATE]: Term 1 election: Requested vote from peers 31346cf196174ddbb2477a9a0aaf83b1 (127.11.118.195:43063), 46cefda75cd24bcdb7c994db3f5c3be1 (127.11.118.194:37317)
I20260929 18:40:47.681301 16237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547cb39c589a4584b20ea633733f5781" candidate_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31346cf196174ddbb2477a9a0aaf83b1"
I20260929 18:40:47.681339 16104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547cb39c589a4584b20ea633733f5781" candidate_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46cefda75cd24bcdb7c994db3f5c3be1"
I20260929 18:40:47.681443 16237 raft_consensus.cc:3037] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:47.681458 16104 raft_consensus.cc:3037] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:47.682809 16104 raft_consensus.cc:2454] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07886a4493a54f7c8d3169adb2a2fbf5 in term 1.
I20260929 18:40:47.682809 16237 raft_consensus.cc:2454] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07886a4493a54f7c8d3169adb2a2fbf5 in term 1.
I20260929 18:40:47.683115 15905 leader_election.cc:302] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07886a4493a54f7c8d3169adb2a2fbf5, 31346cf196174ddbb2477a9a0aaf83b1; no voters:
I20260929 18:40:47.683336 16306 raft_consensus.cc:2781] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:47.683600 16306 raft_consensus.cc:686] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 1 LEADER]: Becoming Leader. State: Replica: 07886a4493a54f7c8d3169adb2a2fbf5, State: Running, Role: LEADER
I20260929 18:40:47.683936 16306 consensus_queue.cc:237] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } }
I20260929 18:40:47.687374 15825 catalog_manager.cc:5495] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07886a4493a54f7c8d3169adb2a2fbf5 (127.11.118.193). New cstate: current_term: 1 leader_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07886a4493a54f7c8d3169adb2a2fbf5" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 46281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:47.706947 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.703434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46766 (local address 127.11.118.193:46281)
0929 18:40:47.703573 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.703577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.703590 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:40:47.703666 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.703668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.703723 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.703817 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:47.704115 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.704626 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.705500 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:47.705717 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:47.705811 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:47.706122 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:47.706619 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:47.706716 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.706771 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.706831 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:40:47.713017 16237 raft_consensus.cc:1261] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 1 FOLLOWER]: Refusing update from remote peer 07886a4493a54f7c8d3169adb2a2fbf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:47.713017 16104 raft_consensus.cc:1261] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 1 FOLLOWER]: Refusing update from remote peer 07886a4493a54f7c8d3169adb2a2fbf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:47.713904 16311 consensus_queue.cc:1035] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31346cf196174ddbb2477a9a0aaf83b1" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 43063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:47.714151 16306 consensus_queue.cc:1035] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46cefda75cd24bcdb7c994db3f5c3be1" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 37317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:47.724613 16316 mvcc.cc:205] Tried to move back new op lower bound from 7334736886624821248 to 7334736886516338688. Current Snapshot: MvccSnapshot[applied={T|T < 7334736886624821248}]
I20260929 18:40:47.729664 15827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:47.729815 15827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:47.732286 15827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f55fd03f1c6404e9a211e3158e8add7: Sending DeleteTablet for 3 replicas of tablet 547cb39c589a4584b20ea633733f5781
I20260929 18:40:47.733201 15951 tablet_service.cc:1514] Processing DeleteTablet for tablet 547cb39c589a4584b20ea633733f5781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:47 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46756
I20260929 18:40:47.733409 16217 tablet_service.cc:1514] Processing DeleteTablet for tablet 547cb39c589a4584b20ea633733f5781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:47 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:33498
I20260929 18:40:47.733421 16084 tablet_service.cc:1514] Processing DeleteTablet for tablet 547cb39c589a4584b20ea633733f5781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:47 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:38174
I20260929 18:40:47.733902 16324 tablet_replica.cc:330] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: stopping tablet replica
I20260929 18:40:47.734027 16323 tablet_replica.cc:330] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5: stopping tablet replica
I20260929 18:40:47.734249 16324 raft_consensus.cc:2229] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:47.734247 16323 raft_consensus.cc:2229] T 547cb39c589a4584b20ea633733f5781 P 07886a4493a54f7c8d3169adb2a2fbf5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:47.734329 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15889
I20260929 18:40:47.735481 16324 raft_consensus.cc:2258] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:47.735982 16325 tablet_replica.cc:330] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: stopping tablet replica
I20260929 18:40:47.736271 16325 raft_consensus.cc:2229] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:47.736515 16325 raft_consensus.cc:2258] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:47.737313 16324 ts_tablet_manager.cc:1905] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:47.737617 16325 ts_tablet_manager.cc:1905] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:47.739888 16325 ts_tablet_manager.cc:1918] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:47.740007 16325 log.cc:1196] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/547cb39c589a4584b20ea633733f5781
I20260929 18:40:47.740348 16325 ts_tablet_manager.cc:1939] T 547cb39c589a4584b20ea633733f5781 P 46cefda75cd24bcdb7c994db3f5c3be1: Deleting consensus metadata
I20260929 18:40:47.740804 16324 ts_tablet_manager.cc:1918] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:47.740916 16324 log.cc:1196] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/547cb39c589a4584b20ea633733f5781
I20260929 18:40:47.741245 16324 ts_tablet_manager.cc:1939] T 547cb39c589a4584b20ea633733f5781 P 31346cf196174ddbb2477a9a0aaf83b1: Deleting consensus metadata
I20260929 18:40:47.741665 15812 catalog_manager.cc:4841] TS 46cefda75cd24bcdb7c994db3f5c3be1 (127.11.118.194:37317): tablet 547cb39c589a4584b20ea633733f5781 (table test-table [id=e26ba8d1107646329f506467ed9b61fb]) successfully deleted
I20260929 18:40:47.742514 15812 catalog_manager.cc:4841] TS 31346cf196174ddbb2477a9a0aaf83b1 (127.11.118.195:43063): tablet 547cb39c589a4584b20ea633733f5781 (table test-table [id=e26ba8d1107646329f506467ed9b61fb]) successfully deleted
I20260929 18:40:47.742801 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16022
W20260929 18:40:47.742889 15813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:46281 (error 108)
I20260929 18:40:47.747309 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.746907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:46281 (local address 127.0.0.1:46776)
0929 18:40:47.747094 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:47.747197 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:46281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 18:40:47.747505 15813 catalog_manager.cc:4584] TS 07886a4493a54f7c8d3169adb2a2fbf5 (127.11.118.193:46281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 547cb39c589a4584b20ea633733f5781: Network error: Client connection negotiation failed: client connection to 127.11.118.193:46281: connect: Connection refused (error 111)
I20260929 18:40:47.748749 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16155
I20260929 18:40:47.756052 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 15796
2026-09-29T18:40:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2351 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T18:40:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:47Z Disabled control of system clock
I20260929 18:40:47.787453 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43239
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:43239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 18:40:47.898227 16330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:47.898540 16330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:47.898577 16330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:47.902516 16330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:47.902570 16330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:47.902585 16330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:47.902601 16330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:47.902616 16330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:47.907373 16330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:43239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:47.908425 16330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:47.909760 16330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:47.918411 16338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.918466 16330 server_base.cc:1034] running on GCE node
W20260929 18:40:47.918414 16336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:47.918418 16335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:47.919217 16330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:47.920310 16330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:47.921502 16330 hybrid_clock.cc:648] HybridClock initialized: now 1790707247921489 us; error 25 us; skew 500 ppm
I20260929 18:40:47.926555 16330 webserver.cc:460] Webserver started at http://127.11.118.254:42703/ using document root <none> and password file <none>
I20260929 18:40:47.927160 16330 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:47.927258 16330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:47.927497 16330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:47.929562 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "0c40e91c7567440e84e7f32762b2470f"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "65950c1996ad4863e16d34337aac46eb"
server_key_iv: "ed589502d2ac888d7a23c6eff4594b83"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.930150 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "0c40e91c7567440e84e7f32762b2470f"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "65950c1996ad4863e16d34337aac46eb"
server_key_iv: "ed589502d2ac888d7a23c6eff4594b83"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.934005 16330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 18:40:47.936549 16344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:47.937810 16330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:47.937984 16330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "0c40e91c7567440e84e7f32762b2470f"
format_stamp: "Formatted at 2026-09-29 18:40:47 on dist-test-slave-cgv7"
server_key: "65950c1996ad4863e16d34337aac46eb"
server_key_iv: "ed589502d2ac888d7a23c6eff4594b83"
server_key_version: "encryptionkey@0"
I20260929 18:40:47.938109 16330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:47.942691 16330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:47.943300 16330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:47.943488 16330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:47.950156 16330 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:43239
I20260929 18:40:47.950183 16396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:43239 every 8 connection(s)
I20260929 18:40:47.951169 16330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:47.953881 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16330
I20260929 18:40:47.954005 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:47.954394 11739 external_mini_cluster.cc:1428] Setting key 4fbf2633bc876249cb471e1950866cc1
I20260929 18:40:47.954581 16397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:47.960350 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:47.955834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.11.118.254:43239)
0929 18:40:47.956507 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:47.956522 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:47.956558 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:40:47.957417 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:47.957438 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:47.957851 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:47.958241 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:47.959056 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:47.959061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:47.959073 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:47.959082 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:47.959135 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:47.959152 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:47.959318 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:47.959392 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 18:40:47.959645 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260929 18:40:47.961624 16397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: Bootstrap starting.
I20260929 18:40:47.963959 16397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:47.964684 16397 log.cc:824] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:47.966671 16397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: No bootstrap required, opened a new log
I20260929 18:40:47.970247 16397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } }
I20260929 18:40:47.970551 16397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:47.970631 16397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c40e91c7567440e84e7f32762b2470f, State: Initialized, Role: FOLLOWER
I20260929 18:40:47.971078 16397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } }
I20260929 18:40:47.971227 16397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:47.971297 16397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:47.971400 16397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:47.972288 16397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } }
I20260929 18:40:47.972630 16397 leader_election.cc:302] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c40e91c7567440e84e7f32762b2470f; no voters:
I20260929 18:40:47.973017 16397 leader_election.cc:288] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:47.973141 16402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:47.973381 16402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [term 1 LEADER]: Becoming Leader. State: Replica: 0c40e91c7567440e84e7f32762b2470f, State: Running, Role: LEADER
I20260929 18:40:47.973742 16402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } }
I20260929 18:40:47.974372 16397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:47.975471 16404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c40e91c7567440e84e7f32762b2470f. Latest consensus state: current_term: 1 leader_uuid: "0c40e91c7567440e84e7f32762b2470f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } } }
I20260929 18:40:47.975907 16404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:47.975796 16403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c40e91c7567440e84e7f32762b2470f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c40e91c7567440e84e7f32762b2470f" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43239 } } }
I20260929 18:40:47.976236 16403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:47.976341 16412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:47.979038 16412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:47.984558 16412 catalog_manager.cc:1347] Generated new cluster ID: cee2f212642d4f37ab089766018f8be0
I20260929 18:40:47.984628 16412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:47.993611 16412 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:47.993944 16412 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:47.998832 16412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: Generated new TSK 0
I20260929 18:40:47.999639 16412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:40:48.007352 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--builtin_ntp_servers=127.11.118.212:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 18:40:48.121582 16421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:48.121924 16421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:48.122020 16421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:48.126318 16421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:48.126413 16421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:48.126518 16421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:48.131866 16421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:48.133157 16421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:48.134647 16421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:48.144704 16426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.144853 16421 server_base.cc:1034] running on GCE node
W20260929 18:40:48.144688 16427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.144764 16429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.145394 16421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:48.146130 16421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:48.147322 16421 hybrid_clock.cc:648] HybridClock initialized: now 1790707248147287 us; error 50 us; skew 500 ppm
I20260929 18:40:48.149325 16421 webserver.cc:460] Webserver started at http://127.11.118.193:41525/ using document root <none> and password file <none>
I20260929 18:40:48.149981 16421 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:48.150085 16421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:48.150373 16421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:48.152587 16421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "4cdaa272131249c1ba29fc32451918ab"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "a09b5bf02eee6e6b1aa7cfd41d1a0dbc"
server_key_iv: "79b34fbb581c1bec354625c507763edd"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.153218 16421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "4cdaa272131249c1ba29fc32451918ab"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "a09b5bf02eee6e6b1aa7cfd41d1a0dbc"
server_key_iv: "79b34fbb581c1bec354625c507763edd"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.157348 16421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:40:48.160212 16435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.161566 16421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:48.161715 16421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "4cdaa272131249c1ba29fc32451918ab"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "a09b5bf02eee6e6b1aa7cfd41d1a0dbc"
server_key_iv: "79b34fbb581c1bec354625c507763edd"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.161849 16421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:48.166975 16421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:48.167665 16421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:48.167872 16421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:48.168452 16421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:48.169404 16421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:48.169476 16421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.169548 16421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:48.169592 16421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.179116 16421 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45001
I20260929 18:40:48.179193 16548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45001 every 8 connection(s)
I20260929 18:40:48.180078 16421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:48.184243 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16421
I20260929 18:40:48.184391 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:48.184698 11739 external_mini_cluster.cc:1428] Setting key 8ab171da04c44441308de5fe37302796
I20260929 18:40:48.187165 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.181927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:47751 (local address 127.11.118.254:43239)
0929 18:40:48.182076 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.182081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.183222 (+ 1141us) server_negotiation.cc:407] Connection header received
0929 18:40:48.184846 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.184849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.184889 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.184968 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.186476 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.186480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.186483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.186497 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.186523 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.186534 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.186653 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.187011 (+ 358us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.187054 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:40:48.187460 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--builtin_ntp_servers=127.11.118.212:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 18:40:48.188392 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.182413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43239 (local address 127.11.118.193:47751)
0929 18:40:48.183070 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.183110 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.184580 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.185184 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.185192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.185310 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.186127 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.186129 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.186171 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.186285 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.186776 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.186790 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.186797 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.187042 (+ 245us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.187311 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":112,"threads_started":1}
I20260929 18:40:48.189769 16549 heartbeater.cc:347] Connected to a master server at 127.11.118.254:43239
I20260929 18:40:48.190030 16549 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:48.190532 16549 heartbeater.cc:511] Master 127.11.118.254:43239 requested a full tablet report, sending...
I20260929 18:40:48.192116 16361 ts_manager.cc:194] Registered new tserver with Master: 4cdaa272131249c1ba29fc32451918ab (127.11.118.193:45001)
W20260929 18:40:48.302145 16552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:48.302534 16552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:48.302610 16552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:48.306856 16552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:48.306968 16552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:48.307070 16552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:48.312037 16552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:48.313215 16552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:48.314677 16552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:48.324630 16558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.324653 16557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.324692 16552 server_base.cc:1034] running on GCE node
W20260929 18:40:48.324649 16560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.325367 16552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:48.325974 16552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:48.327153 16552 hybrid_clock.cc:648] HybridClock initialized: now 1790707248327100 us; error 67 us; skew 500 ppm
I20260929 18:40:48.329053 16552 webserver.cc:460] Webserver started at http://127.11.118.194:41087/ using document root <none> and password file <none>
I20260929 18:40:48.329648 16552 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:48.329712 16552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:48.329967 16552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:48.332104 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "b41029e074b344e8aee32a398fdc97d6"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "25482568cae532ecb68b5b8e794ad445"
server_key_iv: "230d35aa9f8820ef2d07b37aef506257"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.332688 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "b41029e074b344e8aee32a398fdc97d6"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "25482568cae532ecb68b5b8e794ad445"
server_key_iv: "230d35aa9f8820ef2d07b37aef506257"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.336565 16552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:48.339344 16566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.340701 16552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:48.340852 16552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "b41029e074b344e8aee32a398fdc97d6"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "25482568cae532ecb68b5b8e794ad445"
server_key_iv: "230d35aa9f8820ef2d07b37aef506257"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.340981 16552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:48.345768 16552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:48.346446 16552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:48.346625 16552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:48.347167 16552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:48.348073 16552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:48.348142 16552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.348209 16552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:48.348251 16552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.357715 16552 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:43129
I20260929 18:40:48.357735 16679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:43129 every 8 connection(s)
I20260929 18:40:48.358711 16552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:48.364500 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16552
I20260929 18:40:48.364672 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:48.364822 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.360548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:42747 (local address 127.11.118.254:43239)
0929 18:40:48.360683 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.360687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.361706 (+ 1019us) server_negotiation.cc:407] Connection header received
0929 18:40:48.363105 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.363108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.363158 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.363234 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.364290 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.364295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.364298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.364314 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.364337 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.364347 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.364481 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.364599 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.364651 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:40:48.365046 11739 external_mini_cluster.cc:1428] Setting key 0f620f42e0cf18c69ca171a45360fe6f
I20260929 18:40:48.365857 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.360884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43239 (local address 127.11.118.194:42747)
0929 18:40:48.361554 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.361594 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.362874 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.363399 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.363406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.363493 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.364023 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.364024 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.364064 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.364131 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.364456 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.364461 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.364465 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.364597 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.364896 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":110,"threads_started":1}
I20260929 18:40:48.367084 16680 heartbeater.cc:347] Connected to a master server at 127.11.118.254:43239
I20260929 18:40:48.367201 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--builtin_ntp_servers=127.11.118.212:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 18:40:48.367319 16680 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:48.367784 16680 heartbeater.cc:511] Master 127.11.118.254:43239 requested a full tablet report, sending...
I20260929 18:40:48.368995 16361 ts_manager.cc:194] Registered new tserver with Master: b41029e074b344e8aee32a398fdc97d6 (127.11.118.194:43129)
W20260929 18:40:48.483055 16683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:48.483331 16683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:48.483371 16683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:48.487366 16683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:48.487423 16683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:48.487480 16683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:48.492164 16683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:48.493187 16683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:48.494612 16683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:48.504946 16683 server_base.cc:1034] running on GCE node
W20260929 18:40:48.504863 16689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.504863 16691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.504866 16688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.505590 16683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:48.506255 16683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:48.507504 16683 hybrid_clock.cc:648] HybridClock initialized: now 1790707248507485 us; error 30 us; skew 500 ppm
I20260929 18:40:48.509480 16683 webserver.cc:460] Webserver started at http://127.11.118.195:44905/ using document root <none> and password file <none>
I20260929 18:40:48.510071 16683 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:48.510164 16683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:48.510456 16683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:48.512511 16683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "fc4a32276afa4a50acdb13e53f0aaa61"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "91a600f7ae9ecb9ff6eadec23c002e94"
server_key_iv: "5cbf5d5dd7f3c4dfcbe5fa16c77d7ccb"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.513099 16683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "fc4a32276afa4a50acdb13e53f0aaa61"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "91a600f7ae9ecb9ff6eadec23c002e94"
server_key_iv: "5cbf5d5dd7f3c4dfcbe5fa16c77d7ccb"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.516978 16683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 18:40:48.519747 16697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.521118 16683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:48.521267 16683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "fc4a32276afa4a50acdb13e53f0aaa61"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "91a600f7ae9ecb9ff6eadec23c002e94"
server_key_iv: "5cbf5d5dd7f3c4dfcbe5fa16c77d7ccb"
server_key_version: "encryptionkey@0"
I20260929 18:40:48.521405 16683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:48.526245 16683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:48.526961 16683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:48.527148 16683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:48.527698 16683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:48.528611 16683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:48.528683 16683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.528789 16683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:48.528842 16683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:48.537927 16683 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:39497
I20260929 18:40:48.537945 16810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:39497 every 8 connection(s)
I20260929 18:40:48.538962 16683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:48.544472 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16683
I20260929 18:40:48.544593 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:48.544888 11739 external_mini_cluster.cc:1428] Setting key bb8c2add84b4e1b5dcc0f4e8162a04be
I20260929 18:40:48.545616 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.540915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:58791 (local address 127.11.118.254:43239)
0929 18:40:48.541075 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.541080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.542427 (+ 1347us) server_negotiation.cc:407] Connection header received
0929 18:40:48.543776 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.543779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.543828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.543915 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.545120 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.545124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.545127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.545140 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.545171 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.545180 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.545277 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.545433 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.545480 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:40:48.546562 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.541376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43239 (local address 127.11.118.195:58791)
0929 18:40:48.542214 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.542255 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.543554 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.544084 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.544091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.544181 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.544801 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.544803 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.544844 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.544968 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.545269 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.545277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.545283 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.545478 (+ 195us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.545716 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":138,"threads_started":1}
I20260929 18:40:48.547745 16811 heartbeater.cc:347] Connected to a master server at 127.11.118.254:43239
I20260929 18:40:48.548002 16811 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:48.548393 16811 heartbeater.cc:511] Master 127.11.118.254:43239 requested a full tablet report, sending...
I20260929 18:40:48.549363 16361 ts_manager.cc:194] Registered new tserver with Master: fc4a32276afa4a50acdb13e53f0aaa61 (127.11.118.195:39497)
I20260929 18:40:48.559625 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:40:48.563589 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.561715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.11.118.254:43239)
0929 18:40:48.561866 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.561871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.562054 (+ 183us) server_negotiation.cc:407] Connection header received
0929 18:40:48.562162 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.562164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.562200 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.562312 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.563141 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.563145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.563147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.563158 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.563178 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.563186 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.563265 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.563345 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.563382 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:40:48.566834 16361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:40:48.569869 16361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:48.582593 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.580040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.11.118.193:45001)
0929 18:40:48.580560 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.580570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.580590 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:40:48.580640 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.580648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.580800 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.581130 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.581948 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.581956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.581969 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.581980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.582016 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.582042 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.582186 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.582350 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.582421 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":144,"threads_started":1}
I20260929 18:40:48.582621 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.579471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45001 (local address 127.0.0.1:37590)
0929 18:40:48.580115 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.580129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.580190 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.580944 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.580947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.580961 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.581570 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.581572 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.581581 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.581604 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.582191 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.582195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.582200 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.582328 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.582398 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260929 18:40:48.582809 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.579345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.11.118.195:39497)
0929 18:40:48.580286 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.580299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.580322 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:40:48.580377 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.580384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.580591 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.581130 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.581993 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.581997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.582006 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.582015 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.582043 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.582065 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.582540 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.582584 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.582647 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":150,"threads_started":1}
I20260929 18:40:48.582814 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.579190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:39497 (local address 127.0.0.1:45862)
0929 18:40:48.579878 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.579933 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.580031 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.580753 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.580759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.580830 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.581451 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.581456 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.581479 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.581554 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.582167 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.582176 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.582181 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.582317 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.582389 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":108,"threads_started":1}
I20260929 18:40:48.583170 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.579231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:43129 (local address 127.0.0.1:51984)
0929 18:40:48.580040 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.580054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.580132 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.581920 (+ 1788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.581923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.581934 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.582502 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.582504 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.582514 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.582540 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.582943 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.582946 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.582949 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.583021 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.583057 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":98,"threads_started":1}
I20260929 18:40:48.583681 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.580457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.11.118.194:43129)
0929 18:40:48.580921 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.580934 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.580955 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:40:48.581007 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.581011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.581179 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.581312 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.582675 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.582683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.582697 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.582710 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.582752 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.582794 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.582900 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.583452 (+ 552us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.583520 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":108,"threads_started":1}
I20260929 18:40:48.585867 16483 tablet_service.cc:1467] Processing CreateTablet for tablet 74b4c39479804ed6a9c4d88305c74fff (DEFAULT_TABLE table=test-table [id=3e81d4ab566244ddadddfbbffd03e18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:48.585866 16745 tablet_service.cc:1467] Processing CreateTablet for tablet 74b4c39479804ed6a9c4d88305c74fff (DEFAULT_TABLE table=test-table [id=3e81d4ab566244ddadddfbbffd03e18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:48.585937 16614 tablet_service.cc:1467] Processing CreateTablet for tablet 74b4c39479804ed6a9c4d88305c74fff (DEFAULT_TABLE table=test-table [id=3e81d4ab566244ddadddfbbffd03e18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:48.587023 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b4c39479804ed6a9c4d88305c74fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:48.587023 16483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b4c39479804ed6a9c4d88305c74fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:48.587023 16614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b4c39479804ed6a9c4d88305c74fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:48.593377 16828 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Bootstrap starting.
I20260929 18:40:48.594928 16829 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Bootstrap starting.
I20260929 18:40:48.595912 16828 tablet_bootstrap.cc:654] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:48.596771 16828 log.cc:824] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:48.597004 16829 tablet_bootstrap.cc:654] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:48.597525 16830 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Bootstrap starting.
I20260929 18:40:48.597826 16829 log.cc:824] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:48.599073 16828 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: No bootstrap required, opened a new log
I20260929 18:40:48.599370 16828 ts_tablet_manager.cc:1397] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 18:40:48.599686 16830 tablet_bootstrap.cc:654] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:48.600098 16829 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: No bootstrap required, opened a new log
I20260929 18:40:48.600292 16829 ts_tablet_manager.cc:1397] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260929 18:40:48.600458 16830 log.cc:824] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:48.602445 16830 tablet_bootstrap.cc:492] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: No bootstrap required, opened a new log
I20260929 18:40:48.602672 16830 ts_tablet_manager.cc:1397] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:40:48.604549 16828 raft_consensus.cc:348] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.604866 16828 raft_consensus.cc:374] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:48.604758 16829 raft_consensus.cc:348] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.604967 16828 raft_consensus.cc:729] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41029e074b344e8aee32a398fdc97d6, State: Initialized, Role: FOLLOWER
I20260929 18:40:48.605026 16829 raft_consensus.cc:374] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:48.605086 16829 raft_consensus.cc:729] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4a32276afa4a50acdb13e53f0aaa61, State: Initialized, Role: FOLLOWER
I20260929 18:40:48.605552 16828 consensus_queue.cc:260] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.605552 16829 consensus_queue.cc:260] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.606595 16811 heartbeater.cc:503] Master 127.11.118.254:43239 was elected leader, sending a full tablet report...
I20260929 18:40:48.606535 16830 raft_consensus.cc:348] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.606801 16830 raft_consensus.cc:374] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:48.606839 16828 ts_tablet_manager.cc:1428] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260929 18:40:48.606835 16829 ts_tablet_manager.cc:1428] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260929 18:40:48.606848 16830 raft_consensus.cc:729] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdaa272131249c1ba29fc32451918ab, State: Initialized, Role: FOLLOWER
I20260929 18:40:48.607434 16830 consensus_queue.cc:260] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.608058 16680 heartbeater.cc:503] Master 127.11.118.254:43239 was elected leader, sending a full tablet report...
I20260929 18:40:48.608314 16549 heartbeater.cc:503] Master 127.11.118.254:43239 was elected leader, sending a full tablet report...
I20260929 18:40:48.608485 16830 ts_tablet_manager.cc:1428] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260929 18:40:48.609807 16681 tablet.cc:2369] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:40:48.681308 16550 tablet.cc:2369] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:48.734081 16836 raft_consensus.cc:482] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:48.734334 16836 raft_consensus.cc:504] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.735405 16836 leader_election.cc:288] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc4a32276afa4a50acdb13e53f0aaa61 (127.11.118.195:39497), b41029e074b344e8aee32a398fdc97d6 (127.11.118.194:43129)
I20260929 18:40:48.737681 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.735809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39191 (local address 127.11.118.195:39497)
0929 18:40:48.736055 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.736060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.736130 (+ 70us) server_negotiation.cc:407] Connection header received
0929 18:40:48.736264 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.736268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.736312 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.736399 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.737244 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.737247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.737250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.737262 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.737284 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.737294 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.737393 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.737504 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.737546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260929 18:40:48.737681 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.735906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:55495 (local address 127.11.118.194:43129)
0929 18:40:48.736107 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.736114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.736130 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:40:48.736264 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.736268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.736313 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.736399 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.737244 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.737247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.737250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.737262 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.737284 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.737314 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.737400 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.737504 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.737546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260929 18:40:48.737720 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.735680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:43129 (local address 127.11.118.193:55495)
0929 18:40:48.736032 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.736043 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.736157 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.736452 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.736455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.736473 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.737082 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.737084 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.737090 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.737111 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.737399 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.737402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.737406 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.737490 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.737544 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":135,"spinlock_wait_cycles":6400,"thread_start_us":71,"threads_started":1}
I20260929 18:40:48.737723 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.735680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:39497 (local address 127.11.118.193:39191)
0929 18:40:48.736021 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.736038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:48.736157 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:48.736452 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:48.736455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:48.736473 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:48.737065 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:40:48.737069 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:48.737081 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:40:48.737111 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:48.737399 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:48.737402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:48.737406 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 18:40:48.737490 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:40:48.737555 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":18,"spinlock_wait_cycles":23424,"thread_start_us":111,"threads_started":1}
I20260929 18:40:48.738402 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b4c39479804ed6a9c4d88305c74fff" candidate_uuid: "4cdaa272131249c1ba29fc32451918ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41029e074b344e8aee32a398fdc97d6" is_pre_election: true
I20260929 18:40:48.738595 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b4c39479804ed6a9c4d88305c74fff" candidate_uuid: "4cdaa272131249c1ba29fc32451918ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" is_pre_election: true
I20260929 18:40:48.738750 16634 raft_consensus.cc:2454] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cdaa272131249c1ba29fc32451918ab in term 0.
I20260929 18:40:48.738934 16765 raft_consensus.cc:2454] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cdaa272131249c1ba29fc32451918ab in term 0.
I20260929 18:40:48.739252 16436 leader_election.cc:302] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cdaa272131249c1ba29fc32451918ab, b41029e074b344e8aee32a398fdc97d6; no voters:
I20260929 18:40:48.739537 16836 raft_consensus.cc:2781] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:48.739621 16836 raft_consensus.cc:482] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:48.739672 16836 raft_consensus.cc:3037] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:48.740732 16836 raft_consensus.cc:504] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.741180 16836 leader_election.cc:288] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [CANDIDATE]: Term 1 election: Requested vote from peers fc4a32276afa4a50acdb13e53f0aaa61 (127.11.118.195:39497), b41029e074b344e8aee32a398fdc97d6 (127.11.118.194:43129)
I20260929 18:40:48.741683 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b4c39479804ed6a9c4d88305c74fff" candidate_uuid: "4cdaa272131249c1ba29fc32451918ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4a32276afa4a50acdb13e53f0aaa61"
I20260929 18:40:48.741678 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b4c39479804ed6a9c4d88305c74fff" candidate_uuid: "4cdaa272131249c1ba29fc32451918ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41029e074b344e8aee32a398fdc97d6"
I20260929 18:40:48.741886 16765 raft_consensus.cc:3037] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:48.741906 16634 raft_consensus.cc:3037] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:48.743052 16765 raft_consensus.cc:2454] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cdaa272131249c1ba29fc32451918ab in term 1.
I20260929 18:40:48.743086 16634 raft_consensus.cc:2454] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cdaa272131249c1ba29fc32451918ab in term 1.
I20260929 18:40:48.743347 16437 leader_election.cc:302] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cdaa272131249c1ba29fc32451918ab, fc4a32276afa4a50acdb13e53f0aaa61; no voters:
I20260929 18:40:48.743534 16836 raft_consensus.cc:2781] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:48.743829 16836 raft_consensus.cc:686] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 1 LEADER]: Becoming Leader. State: Replica: 4cdaa272131249c1ba29fc32451918ab, State: Running, Role: LEADER
I20260929 18:40:48.744191 16836 consensus_queue.cc:237] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } }
I20260929 18:40:48.747709 16361 catalog_manager.cc:5495] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cdaa272131249c1ba29fc32451918ab (127.11.118.193). New cstate: current_term: 1 leader_uuid: "4cdaa272131249c1ba29fc32451918ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cdaa272131249c1ba29fc32451918ab" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45001 } health_report { overall_health: HEALTHY } } }
I20260929 18:40:48.777354 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.775618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.11.118.193:45001)
0929 18:40:48.775788 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:48.775792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:48.775814 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:40:48.775889 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:48.775892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:48.775952 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:48.776052 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:48.776900 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:48.776903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:48.776906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:48.776924 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:48.776943 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:48.776951 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:48.777048 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:48.777152 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 18:40:48.777197 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:40:48.783641 16765 raft_consensus.cc:1261] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 1 FOLLOWER]: Refusing update from remote peer 4cdaa272131249c1ba29fc32451918ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:48.783641 16634 raft_consensus.cc:1261] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 1 FOLLOWER]: Refusing update from remote peer 4cdaa272131249c1ba29fc32451918ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:48.784457 16839 consensus_queue.cc:1035] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b41029e074b344e8aee32a398fdc97d6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:48.784719 16836 consensus_queue.cc:1035] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4a32276afa4a50acdb13e53f0aaa61" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:40:48.789953 16812 tablet.cc:2369] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:48.796912 16845 mvcc.cc:205] Tried to move back new op lower bound from 7334736891010048000 to 7334736890859565056. Current Snapshot: MvccSnapshot[applied={T|T < 7334736891010048000}]
I20260929 18:40:48.802625 16360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:48.802819 16360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:48.805228 16360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c40e91c7567440e84e7f32762b2470f: Sending DeleteTablet for 3 replicas of tablet 74b4c39479804ed6a9c4d88305c74fff
I20260929 18:40:48.806071 16614 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b4c39479804ed6a9c4d88305c74fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:48 UTC) from {username='slave'} at 127.0.0.1:51984
I20260929 18:40:48.806077 16745 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b4c39479804ed6a9c4d88305c74fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:48 UTC) from {username='slave'} at 127.0.0.1:45862
I20260929 18:40:48.806306 16483 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b4c39479804ed6a9c4d88305c74fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:48 UTC) from {username='slave'} at 127.0.0.1:37590
I20260929 18:40:48.806779 16852 tablet_replica.cc:330] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab: stopping tablet replica
I20260929 18:40:48.807046 16851 tablet_replica.cc:330] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: stopping tablet replica
I20260929 18:40:48.807111 16852 raft_consensus.cc:2229] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:48.807336 16851 raft_consensus.cc:2229] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:48.807484 16852 raft_consensus.cc:2258] T 74b4c39479804ed6a9c4d88305c74fff P 4cdaa272131249c1ba29fc32451918ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:48.807698 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16421
I20260929 18:40:48.807703 16851 raft_consensus.cc:2258] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:48.807884 16853 tablet_replica.cc:330] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: stopping tablet replica
I20260929 18:40:48.808156 16853 raft_consensus.cc:2229] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:48.808506 16853 raft_consensus.cc:2258] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:48.810305 16851 ts_tablet_manager.cc:1905] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:48.810760 16853 ts_tablet_manager.cc:1905] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:48.812829 16853 ts_tablet_manager.cc:1918] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:48.812947 16853 log.cc:1196] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/74b4c39479804ed6a9c4d88305c74fff
I20260929 18:40:48.813249 16853 ts_tablet_manager.cc:1939] T 74b4c39479804ed6a9c4d88305c74fff P b41029e074b344e8aee32a398fdc97d6: Deleting consensus metadata
I20260929 18:40:48.813781 16851 ts_tablet_manager.cc:1918] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:48.813882 16851 log.cc:1196] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/74b4c39479804ed6a9c4d88305c74fff
I20260929 18:40:48.814222 16851 ts_tablet_manager.cc:1939] T 74b4c39479804ed6a9c4d88305c74fff P fc4a32276afa4a50acdb13e53f0aaa61: Deleting consensus metadata
I20260929 18:40:48.814536 16345 catalog_manager.cc:4841] TS b41029e074b344e8aee32a398fdc97d6 (127.11.118.194:43129): tablet 74b4c39479804ed6a9c4d88305c74fff (table test-table [id=3e81d4ab566244ddadddfbbffd03e18b]) successfully deleted
W20260929 18:40:48.815428 16347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:45001 (error 108)
I20260929 18:40:48.815562 16346 catalog_manager.cc:4841] TS fc4a32276afa4a50acdb13e53f0aaa61 (127.11.118.195:39497): tablet 74b4c39479804ed6a9c4d88305c74fff (table test-table [id=3e81d4ab566244ddadddfbbffd03e18b]) successfully deleted
I20260929 18:40:48.816310 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16552
I20260929 18:40:48.816505 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:48.816228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45001 (local address 127.0.0.1:37612)
0929 18:40:48.816336 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:48.816410 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 18:40:48.816974 16347 catalog_manager.cc:4584] TS 4cdaa272131249c1ba29fc32451918ab (127.11.118.193:45001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74b4c39479804ed6a9c4d88305c74fff: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45001: connect: Connection refused (error 111)
I20260929 18:40:48.821703 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16683
I20260929 18:40:48.827214 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16330
2026-09-29T18:40:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1069 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 18:40:48.848791 11739 mini_oidc.cc:152] Starting JWKS server
I20260929 18:40:48.850410 11739 webserver.cc:460] Webserver started at https://127.0.0.1:42451/ using document root <none> and password file <none>
I20260929 18:40:48.850586 11739 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 18:40:48.850888 11739 webserver.cc:460] Webserver started at http://127.0.0.1:45255/ using document root <none> and password file <none>
2026-09-29T18:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:48Z Disabled control of system clock
I20260929 18:40:48.860373 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43257
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:39381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:43257
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42451/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/testchainca.cert with env {}
W20260929 18:40:48.973125 16860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:48.973449 16860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:48.973493 16860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:48.977902 16860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:48.977984 16860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:48.978001 16860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:48.978018 16860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:48.978034 16860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 18:40:48.978051 16860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 18:40:48.978098 16860 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42451/jwks.json
I20260929 18:40:48.983875 16860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:43257
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42451/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:48.985095 16860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:48.986299 16860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:48.995349 16860 server_base.cc:1034] running on GCE node
W20260929 18:40:48.995281 16865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.995507 16868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:48.995368 16866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:48.996228 16860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:48.997419 16860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:48.998648 16860 hybrid_clock.cc:648] HybridClock initialized: now 1790707248998630 us; error 32 us; skew 500 ppm
I20260929 18:40:49.000726 16860 webserver.cc:460] Webserver started at http://127.11.118.254:37703/ using document root <none> and password file <none>
I20260929 18:40:49.001529 16860 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:49.001644 16860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:49.001900 16860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:49.004101 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "e7b2883f4e1346a894d9e6216c673072"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "442bc5f92d68f062f4f206f0ad9cf5a4"
server_key_iv: "38406314b71f34dbddc5ca4dcc4202c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.004690 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "e7b2883f4e1346a894d9e6216c673072"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "442bc5f92d68f062f4f206f0ad9cf5a4"
server_key_iv: "38406314b71f34dbddc5ca4dcc4202c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.008803 16860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:49.011890 16874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:49.013407 16860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:49.013597 16860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e7b2883f4e1346a894d9e6216c673072"
format_stamp: "Formatted at 2026-09-29 18:40:48 on dist-test-slave-cgv7"
server_key: "442bc5f92d68f062f4f206f0ad9cf5a4"
server_key_iv: "38406314b71f34dbddc5ca4dcc4202c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.013732 16860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:49.050755 16860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:49.051582 16860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:49.051797 16860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:49.059090 16860 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:43257
I20260929 18:40:49.059113 16926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:43257 every 8 connection(s)
I20260929 18:40:49.060055 16860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:49.063408 16927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:49.067272 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16860
I20260929 18:40:49.067414 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:49.067728 11739 external_mini_cluster.cc:1428] Setting key 6e01efd30742da48ded82cda87b6df8e
I20260929 18:40:49.069679 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072: Bootstrap starting.
I20260929 18:40:49.071995 16927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:49.072780 16927 log.cc:824] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:49.074716 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072: No bootstrap required, opened a new log
I20260929 18:40:49.077536 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.069207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.11.118.254:43257)
0929 18:40:49.069947 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.069966 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.070004 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:40:49.070730 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.070748 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.071150 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.071512 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:49.072022 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.073111 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.074010 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.074559 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.074737 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.075597 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:49.075603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:49.075615 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:49.076013 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:49.076068 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:49.076088 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:49.076193 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:49.076442 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 18:40:49.076810 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":131,"threads_started":1}
I20260929 18:40:49.078456 16927 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } }
I20260929 18:40:49.078753 16927 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:49.078835 16927 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b2883f4e1346a894d9e6216c673072, State: Initialized, Role: FOLLOWER
I20260929 18:40:49.079288 16927 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } }
I20260929 18:40:49.079442 16927 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:49.079509 16927 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:49.079609 16927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:49.080725 16927 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } }
I20260929 18:40:49.080994 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:40:49.081199 16927 leader_election.cc:302] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7b2883f4e1346a894d9e6216c673072; no voters:
I20260929 18:40:49.081558 16927 leader_election.cc:288] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:49.081681 16932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:49.081894 16932 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [term 1 LEADER]: Becoming Leader. State: Replica: e7b2883f4e1346a894d9e6216c673072, State: Running, Role: LEADER
I20260929 18:40:49.082315 16932 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } }
I20260929 18:40:49.082882 16927 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:49.084160 16934 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b2883f4e1346a894d9e6216c673072. Latest consensus state: current_term: 1 leader_uuid: "e7b2883f4e1346a894d9e6216c673072" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } } }
I20260929 18:40:49.084303 16934 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.084810 16933 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b2883f4e1346a894d9e6216c673072" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b2883f4e1346a894d9e6216c673072" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43257 } } }
I20260929 18:40:49.084942 16933 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.084870 16941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:49.088308 16941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:49.093691 16941 catalog_manager.cc:1347] Generated new cluster ID: 116aee6aacee426da8da613d9c8312c6
I20260929 18:40:49.093760 16941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:49.118131 16941 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:49.119517 16941 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:49.124922 16941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b2883f4e1346a894d9e6216c673072: Generated new TSK 0
I20260929 18:40:49.125769 16941 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 18:40:49.160207 16957 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 18:40:49.160341 16957 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 18:40:49.160784 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.136475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.11.118.254:43257)
0929 18:40:49.136619 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.136624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.136865 (+ 241us) server_negotiation.cc:407] Connection header received
0929 18:40:49.137078 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.137081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.137127 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.137209 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0929 18:40:49.137489 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.138049 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.139317 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.139546 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.139620 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.139994 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 18:40:49.160340 (+ 20346us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 18:40:49.160674 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 18:40:49.160898 16930 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 18:40:49.162204 16956 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:49.136422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43257 (local address 127.0.0.1:52564)
0929 18:40:49.136773 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:49.136789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:49.136925 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:49.137256 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:49.137259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:49.137267 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 18:40:49.137386 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.137392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.138159 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.138162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.139181 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.139192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.139655 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.139659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.139755 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.139807 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 18:40:49.160652 (+ 20845us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 18:40:49.160904 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.118.254:43257: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":125,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.118.254:43257: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 18:40:49.162990 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16860
2026-09-29T18:40:49Z chronyd exiting
I20260929 18:40:49.199796 11739 test_util.cc:175] -----------------------------------------------
I20260929 18:40:49.199878 11739 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790707223158922-11739-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 18:40:49.200860 11739 mini_oidc.cc:152] Starting JWKS server
I20260929 18:40:49.201678 11739 webserver.cc:460] Webserver started at https://127.0.0.1:39971/ using document root <none> and password file <none>
I20260929 18:40:49.201756 11739 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 18:40:49.202009 11739 webserver.cc:460] Webserver started at http://127.0.0.1:40247/ using document root <none> and password file <none>
2026-09-29T18:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:49Z Disabled control of system clock
I20260929 18:40:49.210557 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33321
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:33249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:33321
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39971/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/testchainca.cert with env {}
W20260929 18:40:49.325938 16963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:49.326239 16963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:49.326366 16963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:49.330737 16963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:49.330818 16963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:49.330834 16963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:49.330852 16963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:49.330869 16963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 18:40:49.330886 16963 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 18:40:49.330948 16963 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39971/jwks.json
I20260929 18:40:49.336802 16963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:33321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39971/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:49.337951 16963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:49.339026 16963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:49.348129 16969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:49.348177 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:49.348129 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:49.348170 16963 server_base.cc:1034] running on GCE node
I20260929 18:40:49.349082 16963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:49.350488 16963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:49.351706 16963 hybrid_clock.cc:648] HybridClock initialized: now 1790707249351685 us; error 35 us; skew 500 ppm
I20260929 18:40:49.353821 16963 webserver.cc:460] Webserver started at http://127.11.118.254:35171/ using document root <none> and password file <none>
I20260929 18:40:49.354460 16963 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:49.354549 16963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:49.354771 16963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:49.356868 16963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "e5557687d3af47a7895174179ad2c7ca"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a76aaf910e2c65817611ac9aed605145"
server_key_iv: "d6745aeaa575b77718b33c06e0e3ce8a"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.357429 16963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "e5557687d3af47a7895174179ad2c7ca"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a76aaf910e2c65817611ac9aed605145"
server_key_iv: "d6745aeaa575b77718b33c06e0e3ce8a"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.361249 16963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:40:49.363835 16977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:49.365446 16963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:49.365607 16963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e5557687d3af47a7895174179ad2c7ca"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a76aaf910e2c65817611ac9aed605145"
server_key_iv: "d6745aeaa575b77718b33c06e0e3ce8a"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.365726 16963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:49.377202 16963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:49.377921 16963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:49.378131 16963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:49.385119 16963 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:33321
I20260929 18:40:49.385142 17029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:33321 every 8 connection(s)
I20260929 18:40:49.386154 16963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:49.389529 17030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:49.395357 17030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca: Bootstrap starting.
I20260929 18:40:49.396157 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 16963
I20260929 18:40:49.396310 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:49.396587 11739 external_mini_cluster.cc:1428] Setting key 8d4085bb24064fab5c3b86b0c74a7b6f
I20260929 18:40:49.398231 17030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:49.399016 17030 log.cc:824] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:49.400935 17030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca: No bootstrap required, opened a new log
I20260929 18:40:49.404723 17030 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } }
I20260929 18:40:49.404984 17030 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:49.405048 17030 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5557687d3af47a7895174179ad2c7ca, State: Initialized, Role: FOLLOWER
I20260929 18:40:49.405541 17030 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } }
I20260929 18:40:49.405699 17030 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:49.405788 17030 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:49.405917 17030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:49.406121 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.397655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.11.118.254:33321)
0929 18:40:49.398445 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.398463 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.398510 (+ 47us) server_negotiation.cc:407] Connection header received
0929 18:40:49.399178 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.399202 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.399613 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.399971 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:49.400395 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.401470 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.402495 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.402993 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.403170 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.404137 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:49.404142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:49.404155 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:49.404548 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:49.404604 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:49.404624 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:49.404805 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:49.404946 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 18:40:49.405237 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":133,"threads_started":1}
I20260929 18:40:49.406919 17030 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } }
I20260929 18:40:49.407276 17030 leader_election.cc:302] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5557687d3af47a7895174179ad2c7ca; no voters:
I20260929 18:40:49.407618 17030 leader_election.cc:288] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:49.407694 17035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:49.407891 17035 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [term 1 LEADER]: Becoming Leader. State: Replica: e5557687d3af47a7895174179ad2c7ca, State: Running, Role: LEADER
I20260929 18:40:49.408190 17035 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } }
I20260929 18:40:49.408900 17030 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:49.409173 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:40:49.409739 17037 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5557687d3af47a7895174179ad2c7ca. Latest consensus state: current_term: 1 leader_uuid: "e5557687d3af47a7895174179ad2c7ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } } }
I20260929 18:40:49.409868 17037 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.410177 17036 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5557687d3af47a7895174179ad2c7ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5557687d3af47a7895174179ad2c7ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33321 } } }
I20260929 18:40:49.410293 17036 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.410960 17044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:49.413571 17044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:49.418721 17044 catalog_manager.cc:1347] Generated new cluster ID: 4af75f017a234f11852596142eaefd0f
I20260929 18:40:49.418787 17044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:49.436146 17044 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:49.437265 17044 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:49.448027 17044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5557687d3af47a7895174179ad2c7ca: Generated new TSK 0
I20260929 18:40:49.448915 17044 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 18:40:49.474761 17060 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 18:40:49.474936 17060 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 18:40:49.475627 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.458851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.11.118.254:33321)
0929 18:40:49.458998 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.459003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.459208 (+ 205us) server_negotiation.cc:407] Connection header received
0929 18:40:49.459364 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.459367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.459433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.459564 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0929 18:40:49.459856 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.460708 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.461635 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.461984 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.462089 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.462482 (+ 393us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 18:40:49.475163 (+ 12681us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 18:40:49.475472 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52278: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 18:40:49.475701 17033 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52278: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 18:40:49.475864 17059 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:49.458695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33321 (local address 127.0.0.1:52278)
0929 18:40:49.459115 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:49.459129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:49.459233 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:49.459595 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:49.459597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:49.459606 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 18:40:49.459720 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.459727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.460849 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.460852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.461507 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.461515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.462122 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.462125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.462203 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.462230 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 18:40:49.475462 (+ 13232us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 18:40:49.475636 (+ 174us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.118.254:33321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":189,"threads_started":1}
I20260929 18:40:49.476608 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 16963
2026-09-29T18:40:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 18:40:49.516400 11739 mini_oidc.cc:152] Starting JWKS server
I20260929 18:40:49.517267 11739 webserver.cc:460] Webserver started at https://127.0.0.1:42177/ using document root <none> and password file <none>
I20260929 18:40:49.517350 11739 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 18:40:49.517561 11739 webserver.cc:460] Webserver started at http://127.0.0.1:35149/ using document root <none> and password file <none>
2026-09-29T18:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:49Z Disabled control of system clock
I20260929 18:40:49.526839 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33859
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:33859
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42177/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 18:40:49.636883 17066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:49.637223 17066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:49.637306 17066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:49.641460 17066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:49.641569 17066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:49.641598 17066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:49.641638 17066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:49.641670 17066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 18:40:49.641710 17066 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 18:40:49.641750 17066 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42177/jwks.json
I20260929 18:40:49.646816 17066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:33859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42177/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:49.647992 17066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:49.649492 17066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:49.658255 17074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:49.658262 17071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:49.658342 17066 server_base.cc:1034] running on GCE node
W20260929 18:40:49.658262 17072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:49.659125 17066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:49.660233 17066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:49.661396 17066 hybrid_clock.cc:648] HybridClock initialized: now 1790707249661383 us; error 27 us; skew 500 ppm
I20260929 18:40:49.663312 17066 webserver.cc:460] Webserver started at http://127.11.118.254:41331/ using document root <none> and password file <none>
I20260929 18:40:49.663897 17066 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:49.663960 17066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:49.664170 17066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:49.666255 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "684a272bf68740769259c5ad77dd3f6b"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "580266cc2779f3f3898d9050498c7b6d"
server_key_iv: "45f5f059b5b79210d4e41cfcf84b94f9"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.666899 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "684a272bf68740769259c5ad77dd3f6b"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "580266cc2779f3f3898d9050498c7b6d"
server_key_iv: "45f5f059b5b79210d4e41cfcf84b94f9"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.672355 17066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260929 18:40:49.675616 17080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:49.676964 17066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:49.677078 17066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "684a272bf68740769259c5ad77dd3f6b"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "580266cc2779f3f3898d9050498c7b6d"
server_key_iv: "45f5f059b5b79210d4e41cfcf84b94f9"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.677179 17066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:49.693157 17066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:49.693897 17066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:49.694059 17066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:49.701608 17066 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:33859
I20260929 18:40:49.701640 17132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:33859 every 8 connection(s)
I20260929 18:40:49.702637 17066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:49.705881 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:49.711927 17133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b: Bootstrap starting.
I20260929 18:40:49.712806 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17066
I20260929 18:40:49.712946 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:49.713241 11739 external_mini_cluster.cc:1428] Setting key 72284ce60d53d9d9a3a7ba7a63a65147
I20260929 18:40:49.715299 17133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:49.716104 17133 log.cc:824] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:49.718333 17133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b: No bootstrap required, opened a new log
I20260929 18:40:49.721818 17133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } }
I20260929 18:40:49.722113 17133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:49.722219 17133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 684a272bf68740769259c5ad77dd3f6b, State: Initialized, Role: FOLLOWER
I20260929 18:40:49.722689 17133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } }
I20260929 18:40:49.722847 17133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:49.722939 17133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:49.723057 17133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:49.723196 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.714702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.11.118.254:33859)
0929 18:40:49.715489 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.715505 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.715544 (+ 39us) server_negotiation.cc:407] Connection header received
0929 18:40:49.716304 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.716331 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.716782 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.717141 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:49.717605 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.718712 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.719601 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.720087 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.720273 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.721130 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:49.721135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:49.721148 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:49.721556 (+ 408us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:49.721612 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:49.721632 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:49.721739 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:49.721872 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 18:40:49.722164 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":150,"threads_started":1}
I20260929 18:40:49.724292 17133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } }
I20260929 18:40:49.724655 17133 leader_election.cc:302] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 684a272bf68740769259c5ad77dd3f6b; no voters:
I20260929 18:40:49.724990 17133 leader_election.cc:288] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:49.725104 17138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:49.725337 17138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [term 1 LEADER]: Becoming Leader. State: Replica: 684a272bf68740769259c5ad77dd3f6b, State: Running, Role: LEADER
I20260929 18:40:49.725688 17138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } }
I20260929 18:40:49.726248 17133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:49.726539 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:40:49.727414 17140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 684a272bf68740769259c5ad77dd3f6b. Latest consensus state: current_term: 1 leader_uuid: "684a272bf68740769259c5ad77dd3f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } } }
I20260929 18:40:49.727548 17140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.729049 17139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "684a272bf68740769259c5ad77dd3f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684a272bf68740769259c5ad77dd3f6b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33859 } } }
I20260929 18:40:49.729156 17139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:49.729254 17147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:49.731945 17147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:49.738072 17147 catalog_manager.cc:1347] Generated new cluster ID: 6fa8373f58bf4733b0ee996d40f05c75
I20260929 18:40:49.738179 17147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:49.769593 17147 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:49.770645 17147 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:49.776963 17147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 684a272bf68740769259c5ad77dd3f6b: Generated new TSK 0
I20260929 18:40:49.777806 17147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:40:49.795874 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:49.790186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.11.118.254:33859)
0929 18:40:49.790357 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:49.790363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:49.790532 (+ 169us) server_negotiation.cc:407] Connection header received
0929 18:40:49.790691 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:49.790693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:49.790742 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:49.790832 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0929 18:40:49.791099 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.792040 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.793373 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.793594 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.793674 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.794042 (+ 368us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 18:40:49.795503 (+ 1461us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 18:40:49.795725 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54696: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 18:40:49.795948 17136 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54696: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 18:40:49.797710 17162 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:49.790104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33859 (local address 127.0.0.1:54696)
0929 18:40:49.790435 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:49.790450 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:49.790568 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:49.790869 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:49.790872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:49.790880 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 18:40:49.790993 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.790999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.792176 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.792180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.793247 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:49.793258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:49.793728 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:49.793732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:49.793830 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:49.793864 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 18:40:49.795671 (+ 1807us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 18:40:49.797532 (+ 1861us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.118.254:33859: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":1654,"thread_start_us":107,"threads_started":1}
I20260929 18:40:49.798405 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17066
E20260929 18:40:49.807010 17163 webserver.cc:533] Webserver: sslize failed:
E20260929 18:40:49.807094 17163 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T18:40:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T18:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:49Z Disabled control of system clock
I20260929 18:40:49.845777 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41253
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:41253
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/testchainca.cert with env {}
W20260929 18:40:49.956540 17167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:49.956856 17167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:49.956898 17167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:49.961199 17167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:49.961274 17167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:49.961290 17167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:49.961308 17167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:49.961324 17167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 18:40:49.961341 17167 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 18:40:49.961354 17167 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 18:40:49.967031 17167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:41253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:49.968194 17167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:49.969234 17167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:49.977851 17175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:49.977864 17172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:49.978035 17167 server_base.cc:1034] running on GCE node
W20260929 18:40:49.978227 17173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:49.978889 17167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:49.980020 17167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:49.981184 17167 hybrid_clock.cc:648] HybridClock initialized: now 1790707249981167 us; error 32 us; skew 500 ppm
I20260929 18:40:49.983183 17167 webserver.cc:460] Webserver started at http://127.11.118.254:41377/ using document root <none> and password file <none>
I20260929 18:40:49.983771 17167 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:49.983843 17167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:49.984053 17167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:49.986142 17167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "e142d67b8c0b4c94b027aff87eff57e9"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a109e39ab2ac5bec3a4b15e2118b6465"
server_key_iv: "127dd5c8a784d3990d0139b5e7a9d817"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.986789 17167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "e142d67b8c0b4c94b027aff87eff57e9"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a109e39ab2ac5bec3a4b15e2118b6465"
server_key_iv: "127dd5c8a784d3990d0139b5e7a9d817"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.990502 17167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:40:49.993036 17181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:49.994524 17167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:49.994660 17167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e142d67b8c0b4c94b027aff87eff57e9"
format_stamp: "Formatted at 2026-09-29 18:40:49 on dist-test-slave-cgv7"
server_key: "a109e39ab2ac5bec3a4b15e2118b6465"
server_key_iv: "127dd5c8a784d3990d0139b5e7a9d817"
server_key_version: "encryptionkey@0"
I20260929 18:40:49.994781 17167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:50.017225 17167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:50.018002 17167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:50.018217 17167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:50.025521 17233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:41253 every 8 connection(s)
I20260929 18:40:50.025498 17167 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:41253
I20260929 18:40:50.026743 17167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:50.029700 17234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:50.031962 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17167
I20260929 18:40:50.032099 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:50.032411 11739 external_mini_cluster.cc:1428] Setting key 8b23c9b0988671c610613fc83ba14e4f
I20260929 18:40:50.036058 17234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9: Bootstrap starting.
I20260929 18:40:50.038364 17234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:50.039109 17234 log.cc:824] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:50.041185 17234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9: No bootstrap required, opened a new log
I20260929 18:40:50.041796 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:50.033844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.11.118.254:41253)
0929 18:40:50.034586 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:50.034603 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:50.034638 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:40:50.035276 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:50.035293 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:50.035714 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:50.036062 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:50.036552 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.037512 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.038496 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.038980 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.039159 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:50.040033 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:50.040038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:50.040051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:40:50.040440 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:50.040495 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:40:50.040514 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:50.040625 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:50.040765 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 18:40:50.041018 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":140,"threads_started":1}
I20260929 18:40:50.044735 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:40:50.045316 17234 raft_consensus.cc:348] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } }
I20260929 18:40:50.045564 17234 raft_consensus.cc:374] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:50.045644 17234 raft_consensus.cc:729] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e142d67b8c0b4c94b027aff87eff57e9, State: Initialized, Role: FOLLOWER
I20260929 18:40:50.046114 17234 consensus_queue.cc:260] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } }
I20260929 18:40:50.046268 17234 raft_consensus.cc:388] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:50.046381 17234 raft_consensus.cc:482] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:50.046507 17234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:50.047483 17234 raft_consensus.cc:504] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } }
I20260929 18:40:50.047832 17234 leader_election.cc:302] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e142d67b8c0b4c94b027aff87eff57e9; no voters:
I20260929 18:40:50.048184 17234 leader_election.cc:288] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:50.048254 17239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:50.048519 17239 raft_consensus.cc:686] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [term 1 LEADER]: Becoming Leader. State: Replica: e142d67b8c0b4c94b027aff87eff57e9, State: Running, Role: LEADER
I20260929 18:40:50.048915 17239 consensus_queue.cc:237] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } }
I20260929 18:40:50.049531 17234 sys_catalog.cc:564] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:50.050797 17240 sys_catalog.cc:455] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e142d67b8c0b4c94b027aff87eff57e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } } }
I20260929 18:40:50.050798 17241 sys_catalog.cc:455] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e142d67b8c0b4c94b027aff87eff57e9. Latest consensus state: current_term: 1 leader_uuid: "e142d67b8c0b4c94b027aff87eff57e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e142d67b8c0b4c94b027aff87eff57e9" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41253 } } }
I20260929 18:40:50.051002 17240 sys_catalog.cc:458] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:50.051065 17241 sys_catalog.cc:458] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:50.051913 17250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:50.054903 17250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:50.059981 17250 catalog_manager.cc:1347] Generated new cluster ID: d2b5198b04dd47e2980449b64d657a4f
I20260929 18:40:50.060047 17250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:50.092906 17250 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:50.094198 17250 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:50.102828 17250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e142d67b8c0b4c94b027aff87eff57e9: Generated new TSK 0
I20260929 18:40:50.103644 17250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:40:50.156065 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:50.117487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50732 (local address 127.11.118.254:41253)
0929 18:40:50.117642 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:50.117648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:50.117830 (+ 182us) server_negotiation.cc:407] Connection header received
0929 18:40:50.117999 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:50.118001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:50.118049 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:50.118143 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0929 18:40:50.118419 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.118941 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.120202 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.120437 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.120519 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:50.120875 (+ 356us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 18:40:50.155700 (+ 34825us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 18:40:50.155950 (+ 250us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50732: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 18:40:50.156169 17237 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50732: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 18:40:50.156658 17263 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:50.117385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41253 (local address 127.0.0.1:50732)
0929 18:40:50.117719 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:50.117734 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:50.117865 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:50.118174 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:50.118177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:50.118185 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 18:40:50.118326 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:50.118333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.119072 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.119075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:50.120075 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:50.120086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:50.120572 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:50.120576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:50.120672 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:50.120701 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 18:40:50.155953 (+ 35252us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 18:40:50.156179 (+ 226us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.118.254:41253: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":126,"threads_started":1}
I20260929 18:40:50.157471 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17167
2026-09-29T18:40:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (339 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (126 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:50 dist-test-slave-cgv7 krb5kdc[17275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:50 dist-test-slave-cgv7 krb5kdc[17275](info): set up 2 sockets
Sep 29 18:40:50 dist-test-slave-cgv7 krb5kdc[17275](info): commencing operation
krb5kdc: starting...
W20260929 18:40:51.450894 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:51 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:40:51.581693 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40711
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:40711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:51.701776 17288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:51.702064 17288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:51.702105 17288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:51.706192 17288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:51.706347 17288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:51.706388 17288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:51.706408 17288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:51.706423 17288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:51.712035 17288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:40711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:51.713135 17288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:51.714146 17288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:51.723080 17288 server_base.cc:1034] running on GCE node
W20260929 18:40:51.723042 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:51.723042 17293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:51.723061 17296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:51.723862 17288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:51.725062 17288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:51.726241 17288 hybrid_clock.cc:648] HybridClock initialized: now 1790707251726227 us; error 31 us; skew 500 ppm
Sep 29 18:40:51 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707251, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:51.729741 17288 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:40:51.730908 17288 webserver.cc:460] Webserver started at http://127.11.118.254:33103/ using document root <none> and password file <none>
I20260929 18:40:51.731498 17288 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:51.731561 17288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:51.731796 17288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:51.733948 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "1940ac5390bf4dbe94015e17f8937e60"
format_stamp: "Formatted at 2026-09-29 18:40:51 on dist-test-slave-cgv7"
server_key: "14aff37c361891959e9effecc834b11e"
server_key_iv: "36386375b207aacc51a51068724713c4"
server_key_version: "encryptionkey@0"
I20260929 18:40:51.734606 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "1940ac5390bf4dbe94015e17f8937e60"
format_stamp: "Formatted at 2026-09-29 18:40:51 on dist-test-slave-cgv7"
server_key: "14aff37c361891959e9effecc834b11e"
server_key_iv: "36386375b207aacc51a51068724713c4"
server_key_version: "encryptionkey@0"
I20260929 18:40:51.738613 17288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:40:51.741210 17303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:51.742708 17288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:51.742885 17288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "1940ac5390bf4dbe94015e17f8937e60"
format_stamp: "Formatted at 2026-09-29 18:40:51 on dist-test-slave-cgv7"
server_key: "14aff37c361891959e9effecc834b11e"
server_key_iv: "36386375b207aacc51a51068724713c4"
server_key_version: "encryptionkey@0"
I20260929 18:40:51.743011 17288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:51.763011 17288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:51.767365 17288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:51.767591 17288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:51.777190 17288 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:40711
I20260929 18:40:51.777251 17355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:40711 every 8 connection(s)
I20260929 18:40:51.778744 17288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:51.779079 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17288
I20260929 18:40:51.779211 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:51.779606 11739 external_mini_cluster.cc:1428] Setting key 3e85d9561c32bbbfb4b4d5c6e21e9b34
I20260929 18:40:51.781975 17356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 18:40:51 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:51.791076 17356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60: Bootstrap starting.
I20260929 18:40:51.794636 17356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:51.795701 17356 log.cc:824] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:51.795998 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:51.781313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.11.118.254:40711)
0929 18:40:51.781783 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:51.781820 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:51.781863 (+ 43us) server_negotiation.cc:407] Connection header received
0929 18:40:51.782790 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:51.782822 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:51.783319 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:51.783747 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:51.784649 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:51.785683 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:51.786803 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:51.787576 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:51.787744 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:51.790472 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:51.790478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:51.790494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:51.790528 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:51.792550 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:51.793081 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:51.793087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:51.793101 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:51.793223 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:51.793546 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:51.793549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:51.793551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:51.794001 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:51.794214 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:51.794560 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 18:40:51.794906 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":165,"threads_started":1}
I20260929 18:40:51.798128 17356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60: No bootstrap required, opened a new log
I20260929 18:40:51.802161 17356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } }
I20260929 18:40:51.802557 17356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:51.802608 17356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1940ac5390bf4dbe94015e17f8937e60, State: Initialized, Role: FOLLOWER
I20260929 18:40:51.803056 17356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } }
I20260929 18:40:51.803232 17356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:51.803325 17356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:51.803460 17356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:51.804484 17356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } }
I20260929 18:40:51.804793 17356 leader_election.cc:302] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1940ac5390bf4dbe94015e17f8937e60; no voters:
I20260929 18:40:51.805068 17356 leader_election.cc:288] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:51.805214 17361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:51.805475 17361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [term 1 LEADER]: Becoming Leader. State: Replica: 1940ac5390bf4dbe94015e17f8937e60, State: Running, Role: LEADER
I20260929 18:40:51.805805 17361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } }
I20260929 18:40:51.806427 17356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:51.807672 17362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1940ac5390bf4dbe94015e17f8937e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } } }
I20260929 18:40:51.807670 17363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1940ac5390bf4dbe94015e17f8937e60. Latest consensus state: current_term: 1 leader_uuid: "1940ac5390bf4dbe94015e17f8937e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1940ac5390bf4dbe94015e17f8937e60" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40711 } } }
I20260929 18:40:51.807834 17362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:51.807833 17363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:51.808308 17370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:51.811436 17370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:51.816982 17370 catalog_manager.cc:1347] Generated new cluster ID: 203aa5b63cab421a8f89fcc993331fe7
I20260929 18:40:51.817083 17370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:51.828739 17370 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:51.830065 17370 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:51.842572 17370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1940ac5390bf4dbe94015e17f8937e60: Generated new TSK 0
I20260929 18:40:51.843487 17370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:40:51.877183 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:51.991019 17382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:51.991348 17382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:51.991425 17382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:51.995584 17382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:51.995675 17382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:51.995769 17382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:52.001127 17382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:52.002369 17382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:52.003476 17382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:52.013610 17388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:52.013616 17387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:52.013795 17382 server_base.cc:1034] running on GCE node
W20260929 18:40:52.013616 17390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:52.014390 17382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:52.015085 17382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:52.016271 17382 hybrid_clock.cc:648] HybridClock initialized: now 1790707252016254 us; error 62 us; skew 500 ppm
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.019260 17382 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:40:52.020169 17382 webserver.cc:460] Webserver started at http://127.11.118.193:41829/ using document root <none> and password file <none>
I20260929 18:40:52.020746 17382 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:52.020809 17382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:52.021056 17382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:52.023257 17382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "6c7bde7d7b3449a5bf23c1fb199a9a61"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "9d69e29d4e23722118ac4afe14e6736a"
server_key_iv: "0c41f16798b56a1affe0958487b9b742"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.023864 17382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "6c7bde7d7b3449a5bf23c1fb199a9a61"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "9d69e29d4e23722118ac4afe14e6736a"
server_key_iv: "0c41f16798b56a1affe0958487b9b742"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.027833 17382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:52.030443 17397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.031766 17382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:40:52.031905 17382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "6c7bde7d7b3449a5bf23c1fb199a9a61"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "9d69e29d4e23722118ac4afe14e6736a"
server_key_iv: "0c41f16798b56a1affe0958487b9b742"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.032020 17382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:52.051497 17382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:52.055015 17382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:52.055229 17382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:52.055836 17382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:52.056669 17382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:52.056751 17382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.056823 17382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:52.056878 17382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.065891 17382 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:35729
I20260929 18:40:52.065907 17510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:35729 every 8 connection(s)
I20260929 18:40:52.066876 17382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:52.074576 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17382
I20260929 18:40:52.074738 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:52.075048 11739 external_mini_cluster.cc:1428] Setting key b743c8b76409580b328660d43ecc5940
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:52.082062 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.068695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:54687 (local address 127.11.118.254:40711)
0929 18:40:52.068908 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.068912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.069837 (+ 925us) server_negotiation.cc:407] Connection header received
0929 18:40:52.071221 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.071224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.071271 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.071348 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.072620 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.073193 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.074607 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.074963 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.075057 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.078032 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.078035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.078037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.078105 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.079912 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.080450 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.080455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.080456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.080540 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.080897 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.080901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.080903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.081099 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.081209 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.081752 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.081889 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260929 18:40:52.083045 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.069193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40711 (local address 127.11.118.193:54687)
0929 18:40:52.069639 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:52.069681 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:52.070997 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:52.071520 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:52.071526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:52.071950 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:52.072465 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.072479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.073341 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.073344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.074453 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.074466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.075135 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.075140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.075342 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.076039 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:52.076064 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:52.077839 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:52.080034 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.080041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.080060 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.080342 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.080635 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.080638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.080641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.080799 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.081229 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:52.081236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:52.081508 (+ 272us) client_negotiation.cc:769] Sending connection context
0929 18:40:52.081710 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 18:40:52.082078 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":123,"threads_started":1}
I20260929 18:40:52.084646 17511 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40711
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:52.085038 17511 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:52.085685 17511 heartbeater.cc:511] Master 127.11.118.254:40711 requested a full tablet report, sending...
I20260929 18:40:52.087402 17320 ts_manager.cc:194] Registered new tserver with Master: 6c7bde7d7b3449a5bf23c1fb199a9a61 (127.11.118.193:35729)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:52.088688 17320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:54687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:40:52.106140 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:52.217613 17516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:52.217908 17516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:52.217953 17516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:52.222016 17516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:52.222069 17516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:52.222126 17516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:52.227522 17516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:52.228574 17516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:52.229547 17516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:52.239213 17521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:52.239321 17516 server_base.cc:1034] running on GCE node
W20260929 18:40:52.239220 17522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:52.239511 17524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:52.239915 17516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:52.240667 17516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:52.241958 17516 hybrid_clock.cc:648] HybridClock initialized: now 1790707252241915 us; error 53 us; skew 500 ppm
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.245046 17516 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:40:52.246358 17516 webserver.cc:460] Webserver started at http://127.11.118.194:44393/ using document root <none> and password file <none>
I20260929 18:40:52.246956 17516 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:52.247020 17516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:52.247193 17516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:52.249333 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "46d4c88b6ac1409bb4331114ff694a59"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "d1f534846136771eb4cfd6a1631f3b41"
server_key_iv: "1aeb2694150eafeb86d137ac4752ea66"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.249933 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "46d4c88b6ac1409bb4331114ff694a59"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "d1f534846136771eb4cfd6a1631f3b41"
server_key_iv: "1aeb2694150eafeb86d137ac4752ea66"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.253930 17516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:40:52.256659 17531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.257999 17516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:52.258262 17516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "46d4c88b6ac1409bb4331114ff694a59"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "d1f534846136771eb4cfd6a1631f3b41"
server_key_iv: "1aeb2694150eafeb86d137ac4752ea66"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.258406 17516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:52.273440 17516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:52.276424 17516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:52.276633 17516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:52.277261 17516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:52.278093 17516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:52.278141 17516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.278185 17516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:52.278254 17516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.287796 17516 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:43147
I20260929 18:40:52.287807 17644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:43147 every 8 connection(s)
I20260929 18:40:52.288774 17516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:52.293045 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17516
I20260929 18:40:52.293157 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:52.293443 11739 external_mini_cluster.cc:1428] Setting key fbdf1eae4b1c5d349ee5fc8b4935116b
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:40:52.304577 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.290800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:33169 (local address 127.11.118.254:40711)
0929 18:40:52.290991 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.290995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.291906 (+ 911us) server_negotiation.cc:407] Connection header received
0929 18:40:52.293352 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.293356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.293414 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.293529 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.295067 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.295610 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.297271 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.297667 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.297791 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.300766 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.300769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.300774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.300805 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.302356 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.302921 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.302925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.302926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.303011 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.303355 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.303360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.303362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.303567 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.303690 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.304325 (+ 635us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.304447 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 18:40:52.305533 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.291227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40711 (local address 127.11.118.194:33169)
0929 18:40:52.291744 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:52.291788 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:52.293073 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:52.293731 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:52.293738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:52.294116 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:52.294866 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.294927 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.295764 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.295770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.297145 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.297155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.297866 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.297870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.298074 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.298783 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:52.298810 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:52.300574 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:52.302491 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.302497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.302515 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.302792 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.303112 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.303115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.303117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.303258 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.303715 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:52.303724 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:52.304059 (+ 335us) client_negotiation.cc:769] Sending connection context
0929 18:40:52.304247 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 18:40:52.304524 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:52.307134 17645 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40711
I20260929 18:40:52.307462 17645 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:52.308112 17645 heartbeater.cc:511] Master 127.11.118.254:40711 requested a full tablet report, sending...
I20260929 18:40:52.309315 17320 ts_manager.cc:194] Registered new tserver with Master: 46d4c88b6ac1409bb4331114ff694a59 (127.11.118.194:43147)
I20260929 18:40:52.309930 17320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:33169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:40:52.326241 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:52.439471 17650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:52.439798 17650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:52.439895 17650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:52.444160 17650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:52.444257 17650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:52.444510 17650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:52.451828 17650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:52.453042 17650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:52.454124 17650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:52.464476 17650 server_base.cc:1034] running on GCE node
W20260929 18:40:52.464389 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:52.464385 17658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:52.464385 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:52.465104 17650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:52.465752 17650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:52.466949 17650 hybrid_clock.cc:648] HybridClock initialized: now 1790707252466933 us; error 27 us; skew 500 ppm
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.469798 17650 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:40:52.470791 17650 webserver.cc:460] Webserver started at http://127.11.118.195:42483/ using document root <none> and password file <none>
I20260929 18:40:52.471386 17650 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:52.471479 17650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:52.471773 17650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:52.474066 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "438a297644474951b1b52bb3508c7a36"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "0c6bbda2fa0c42f340faf9ac48e9d5cb"
server_key_iv: "2493f2db3ed06b01bbed0cf9dbd5d9e4"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.474735 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "438a297644474951b1b52bb3508c7a36"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "0c6bbda2fa0c42f340faf9ac48e9d5cb"
server_key_iv: "2493f2db3ed06b01bbed0cf9dbd5d9e4"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.478672 17650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:52.481225 17665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.482590 17650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:52.482715 17650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "438a297644474951b1b52bb3508c7a36"
format_stamp: "Formatted at 2026-09-29 18:40:52 on dist-test-slave-cgv7"
server_key: "0c6bbda2fa0c42f340faf9ac48e9d5cb"
server_key_iv: "2493f2db3ed06b01bbed0cf9dbd5d9e4"
server_key_version: "encryptionkey@0"
I20260929 18:40:52.482836 17650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:52.501331 17650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:52.504827 17650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:52.505048 17650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:52.505679 17650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:52.506628 17650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:52.506707 17650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.506789 17650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:52.506834 17650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:52.516649 17650 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:38545
I20260929 18:40:52.516691 17778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:38545 every 8 connection(s)
I20260929 18:40:52.517570 17650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:52.523165 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17650
I20260929 18:40:52.523274 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:52.523582 11739 external_mini_cluster.cc:1428] Setting key 26419788d02668d96ad0d38662c3ffe1
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:40:52.532889 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.519624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:52129 (local address 127.11.118.254:40711)
0929 18:40:52.519777 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.519781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.520741 (+ 960us) server_negotiation.cc:407] Connection header received
0929 18:40:52.522324 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.522329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.522393 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.522472 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.523812 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.524528 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.525980 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.526285 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.526420 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.529078 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.529082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.529084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.529112 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.530831 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.531379 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.531382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.531384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.531461 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.531774 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.531776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.531778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.531993 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.532067 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.532627 (+ 560us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.532743 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:40:52.533825 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.520012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40711 (local address 127.11.118.195:52129)
0929 18:40:52.520528 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:52.520614 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:52.522045 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:52.522622 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:52.522629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:52.523006 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:52.523606 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.523649 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.524699 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.524705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.525812 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:52.525823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.526455 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.526458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:52.526617 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.527271 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:52.527289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:52.528870 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:52.530938 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.530943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.530958 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.531285 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.531549 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:52.531551 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:52.531553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:52.531695 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:52.532091 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:52.532097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:52.532403 (+ 306us) client_negotiation.cc:769] Sending connection context
0929 18:40:52.532575 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 18:40:52.532902 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":167,"threads_started":1}
I20260929 18:40:52.535070 17779 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40711
I20260929 18:40:52.535343 17779 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:52.535931 17779 heartbeater.cc:511] Master 127.11.118.254:40711 requested a full tablet report, sending...
I20260929 18:40:52.537111 17319 ts_manager.cc:194] Registered new tserver with Master: 438a297644474951b1b52bb3508c7a36 (127.11.118.195:38545)
I20260929 18:40:52.537781 17319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:52129
I20260929 18:40:52.538002 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/test-admin.keytab.
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.555493 11739 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 18:40:52.558375 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:52.556899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.11.118.193:35729)
0929 18:40:52.557271 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.557286 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.557330 (+ 44us) server_negotiation.cc:407] Connection header received
0929 18:40:52.557379 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.557384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.557573 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.557683 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.558151 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54170: BlockingRecv error: recv got EOF from 127.0.0.1:54170 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":135,"threads_started":1}
W20260929 18:40:52.558827 11739 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc)
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.559896 11739 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.560865 11739 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:52.571192 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.561916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.11.118.193:35729)
0929 18:40:52.562065 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.562069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.562167 (+ 98us) server_negotiation.cc:407] Connection header received
0929 18:40:52.562387 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.562390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.562445 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.562535 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.563131 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.563914 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.564753 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.565163 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.565279 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.567644 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.567652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.567666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.567690 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.569317 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.569786 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.569792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.569793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.569873 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.570150 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.570153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.570155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.570475 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.570690 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.570890 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.571041 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 18:40:52.575315 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:52.574180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.11.118.193:35729)
0929 18:40:52.574365 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.574368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.574431 (+ 63us) server_negotiation.cc:407] Connection header received
0929 18:40:52.574523 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.574525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.574563 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.574631 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.575183 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54192: BlockingRecv error: recv got EOF from 127.0.0.1:54192 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 18:40:52.575321 11739 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc)
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.576601 11739 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.577785 11739 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:52.587240 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.579017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.11.118.193:35729)
0929 18:40:52.579141 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.579145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.579158 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:40:52.579193 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.579195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.579232 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.579306 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.579950 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.580480 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.581325 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.581551 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.581638 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.584067 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.584070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.584072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.584097 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.585409 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.585884 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.585889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.585891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.585971 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.586255 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.586258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.586260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.586491 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.586631 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.586906 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.587027 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 18:40:52.590868 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:40:52.589358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.11.118.193:35729)
0929 18:40:52.589564 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.589568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.589820 (+ 252us) server_negotiation.cc:407] Connection header received
0929 18:40:52.590086 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.590089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.590142 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.590231 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.590748 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54196: BlockingRecv error: recv got EOF from 127.0.0.1:54196 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260929 18:40:52.591153 11739 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790707223158922-11739-0/krb5kdc/krb5cc' of type FILE
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.592293 11739 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:52.593284 11739 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:40:52.602667 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:52.594371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.11.118.193:35729)
0929 18:40:52.594494 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:52.594497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:52.594680 (+ 183us) server_negotiation.cc:407] Connection header received
0929 18:40:52.594735 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:52.594737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:52.594829 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:52.594914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:52.595460 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.595997 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.596870 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:52.597084 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:52.597166 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:52.599598 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:52.599601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:52.599603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:52.599629 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:52.601101 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.601509 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.601511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.601513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.601584 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:52.601871 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:52.601873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:52.601875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:52.602036 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:52.602130 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:52.602433 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 18:40:52.602541 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:40:52.603950 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17382
I20260929 18:40:52.610203 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17516
I20260929 18:40:52.615655 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17650
I20260929 18:40:52.620759 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17288
2026-09-29T18:40:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2210 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17799](info): set up 2 sockets
Sep 29 18:40:52 dist-test-slave-cgv7 krb5kdc[17799](info): commencing operation
krb5kdc: starting...
W20260929 18:40:53.671950 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:40:53 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:40:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:40:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab.
Entry for principal oryx/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab.
I20260929 18:40:53.797514 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39579
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab
--principal=oryx/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:53.909775 17812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:53.910094 17812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:53.910137 17812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:53.914386 17812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:53.914462 17812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:53.914482 17812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:53.914500 17812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:53.914516 17812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:53.920195 17812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab
--principal=oryx/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:53.921401 17812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:53.922542 17812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:40:53.932098 17812 server_base.cc:1034] running on GCE node
W20260929 18:40:53.932040 17818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:53.932035 17820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:53.932040 17817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:53.932965 17812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:53.934124 17812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:53.935338 17812 hybrid_clock.cc:648] HybridClock initialized: now 1790707253935309 us; error 41 us; skew 500 ppm
Sep 29 18:40:53 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:53.943852 17812 init.cc:377] Logged in from keytab as oryx/127.11.118.254@KRBTEST.COM (short username oryx)
I20260929 18:40:53.944914 17812 webserver.cc:460] Webserver started at http://127.11.118.254:35933/ using document root <none> and password file <none>
I20260929 18:40:53.945524 17812 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:53.945621 17812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:53.945941 17812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:53.948103 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "71f51d3eac064c56a2d30ff52c498988"
format_stamp: "Formatted at 2026-09-29 18:40:53 on dist-test-slave-cgv7"
server_key: "e89f915f80448709f92c25462ad3f083"
server_key_iv: "56bca32ac813ebb023cdec7bf4b94367"
server_key_version: "encryptionkey@0"
I20260929 18:40:53.948704 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "71f51d3eac064c56a2d30ff52c498988"
format_stamp: "Formatted at 2026-09-29 18:40:53 on dist-test-slave-cgv7"
server_key: "e89f915f80448709f92c25462ad3f083"
server_key_iv: "56bca32ac813ebb023cdec7bf4b94367"
server_key_version: "encryptionkey@0"
I20260929 18:40:53.952615 17812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:53.955488 17827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:53.956923 17812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:53.957094 17812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "71f51d3eac064c56a2d30ff52c498988"
format_stamp: "Formatted at 2026-09-29 18:40:53 on dist-test-slave-cgv7"
server_key: "e89f915f80448709f92c25462ad3f083"
server_key_iv: "56bca32ac813ebb023cdec7bf4b94367"
server_key_version: "encryptionkey@0"
I20260929 18:40:53.957221 17812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:53.976217 17812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:53.979276 17812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:53.979498 17812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:53.987059 17812 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39579
I20260929 18:40:53.987382 17879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39579 every 8 connection(s)
I20260929 18:40:53.988101 17812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:40:53.990949 17880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:53.994546 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17812
I20260929 18:40:53.994700 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:40:53.995008 11739 external_mini_cluster.cc:1428] Setting key c2b5bb75aa6ead23d3060f6c00f9daa9
I20260929 18:40:53.997287 17880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Bootstrap starting.
I20260929 18:40:53.999627 17880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:54.000394 17880 log.cc:824] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:54.002398 17880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: No bootstrap required, opened a new log
Sep 29 18:40:53 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:40:54.006870 17880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } }
I20260929 18:40:54.007200 17880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:54.007270 17880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f51d3eac064c56a2d30ff52c498988, State: Initialized, Role: FOLLOWER
I20260929 18:40:54.007848 17880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } }
I20260929 18:40:54.008031 17880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:40:54.008131 17880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:40:54.008237 17880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:54.009490 17880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } }
I20260929 18:40:54.009857 17880 leader_election.cc:302] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71f51d3eac064c56a2d30ff52c498988; no voters:
I20260929 18:40:54.010183 17880 leader_election.cc:288] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:40:54.010327 17885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:54.010398 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:53.996518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.11.118.254:39579)
0929 18:40:53.996996 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:53.997011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:53.997048 (+ 37us) server_negotiation.cc:407] Connection header received
0929 18:40:53.997814 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:53.997833 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:53.998169 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:53.998572 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:53.999390 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.000429 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.001333 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.001819 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.002001 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.004861 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.004866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.004881 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.004910 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:54.006907 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.007500 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.007507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.007513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.007656 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.007970 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.007973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.007974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.008555 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:54.008718 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:54.009023 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 18:40:54.009359 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":143,"threads_started":1}
I20260929 18:40:54.010563 17885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 LEADER]: Becoming Leader. State: Replica: 71f51d3eac064c56a2d30ff52c498988, State: Running, Role: LEADER
I20260929 18:40:54.010941 17885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } }
I20260929 18:40:54.011492 17880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:54.013911 17887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f51d3eac064c56a2d30ff52c498988. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } }
I20260929 18:40:54.013842 17886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } }
I20260929 18:40:54.013994 17887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:54.013994 17886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
W20260929 18:40:54.016719 17899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:54.016813 17899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:54.016872 17900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:54.017694 17900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:54.023312 17900 catalog_manager.cc:1347] Generated new cluster ID: 7baea67fa1604e70b6ce11601db023ab
I20260929 18:40:54.023391 17900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:54.039712 17900 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:40:54.040989 17900 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:54.049173 17900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Generated new TSK 0
I20260929 18:40:54.050016 17900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
I20260929 18:40:54.077682 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:54.191779 17906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:54.192111 17906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:54.192193 17906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:54.196754 17906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:54.196887 17906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:54.196990 17906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:54.202610 17906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:54.203747 17906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:54.204794 17906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:54.214615 17912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.214608 17911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.214604 17914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:54.214943 17906 server_base.cc:1034] running on GCE node
I20260929 18:40:54.215415 17906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:54.216075 17906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:54.217245 17906 hybrid_clock.cc:648] HybridClock initialized: now 1790707254217228 us; error 30 us; skew 500 ppm
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:54.220361 17906 init.cc:377] Logged in from keytab as oryx/127.11.118.193@KRBTEST.COM (short username oryx)
I20260929 18:40:54.221325 17906 webserver.cc:460] Webserver started at http://127.11.118.193:45525/ using document root <none> and password file <none>
I20260929 18:40:54.221911 17906 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:54.222007 17906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:54.222252 17906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:54.224473 17906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "f54790de3458470ba4da284f400e5bc8"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "2766a2f6068487d434048fc25aae7f61"
server_key_iv: "89600c5c954a7f55a08953fb5338c160"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.225075 17906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "f54790de3458470ba4da284f400e5bc8"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "2766a2f6068487d434048fc25aae7f61"
server_key_iv: "89600c5c954a7f55a08953fb5338c160"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.229102 17906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:40:54.231796 17921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.233103 17906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:54.233256 17906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "f54790de3458470ba4da284f400e5bc8"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "2766a2f6068487d434048fc25aae7f61"
server_key_iv: "89600c5c954a7f55a08953fb5338c160"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.233383 17906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:54.265363 17906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:54.268597 17906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:54.268829 17906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:54.269436 17906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:54.270627 17906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:54.270709 17906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.270813 17906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:54.270866 17906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.279979 17906 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:42325
I20260929 18:40:54.280035 18034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:42325 every 8 connection(s)
I20260929 18:40:54.280936 17906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:54.284770 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 17906
I20260929 18:40:54.284924 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:40:54.285218 11739 external_mini_cluster.cc:1428] Setting key 0d4c88dc2caeadfe1e2ea5e87084554b
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:40:54.296487 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.282828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:55001 (local address 127.11.118.254:39579)
0929 18:40:54.282991 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:54.282995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:54.283861 (+ 866us) server_negotiation.cc:407] Connection header received
0929 18:40:54.285357 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:54.285362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:54.285425 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:54.285534 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:54.286834 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.287370 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.289016 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.289250 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.289347 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.292124 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.292128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.292130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.292156 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:54.293924 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.294660 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.294668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.294669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.294752 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.295147 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.295151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.295152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.295369 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:54.295486 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:54.296138 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 18:40:54.296289 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.194@KRBTEST.COM" created.
I20260929 18:40:54.297562 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.283198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:55001)
0929 18:40:54.283686 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:54.283732 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:54.285106 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:54.285722 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:54.285730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:54.286098 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:54.286686 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.286701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.287508 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.287516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.288862 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.288877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.289427 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.289430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.289635 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.290377 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:54.290404 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:54.291919 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:54.294158 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.294164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.294182 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.294543 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.294863 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.294866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.294868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.295041 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.295486 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:54.295494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:54.295861 (+ 367us) client_negotiation.cc:769] Sending connection context
0929 18:40:54.296085 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 18:40:54.296436 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":115,"threads_started":1}
I20260929 18:40:54.299517 18035 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:54.299790 18035 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:54.300458 18035 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
I20260929 18:40:54.305341 17844 ts_manager.cc:194] Registered new tserver with Master: f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325)
I20260929 18:40:54.307289 17844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:55001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
I20260929 18:40:54.322829 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:54.436622 18040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:54.436937 18040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:54.437011 18040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:54.441329 18040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:54.441403 18040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:54.441507 18040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:54.446966 18040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:54.448122 18040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:54.449275 18040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:54.459545 18046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.459581 18048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.459748 18045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:54.459789 18040 server_base.cc:1034] running on GCE node
I20260929 18:40:54.460661 18040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:54.461462 18040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:54.462615 18040 hybrid_clock.cc:648] HybridClock initialized: now 1790707254462582 us; error 46 us; skew 500 ppm
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:54.465653 18040 init.cc:377] Logged in from keytab as oryx/127.11.118.194@KRBTEST.COM (short username oryx)
I20260929 18:40:54.466626 18040 webserver.cc:460] Webserver started at http://127.11.118.194:38729/ using document root <none> and password file <none>
I20260929 18:40:54.467221 18040 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:54.467288 18040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:54.467511 18040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:54.469708 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "71bdf123505e4677a00bdf8b2e1e2201"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "c7103b8156f4d9525b4ca28d82aca205"
server_key_iv: "9d9db806c136403fbc5f6fd03be77802"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.470391 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "71bdf123505e4677a00bdf8b2e1e2201"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "c7103b8156f4d9525b4ca28d82aca205"
server_key_iv: "9d9db806c136403fbc5f6fd03be77802"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.474985 18040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:40:54.477494 18055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.478866 18040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:54.479005 18040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "71bdf123505e4677a00bdf8b2e1e2201"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "c7103b8156f4d9525b4ca28d82aca205"
server_key_iv: "9d9db806c136403fbc5f6fd03be77802"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.479116 18040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:54.493291 18040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:54.496562 18040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:54.496716 18040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:54.497244 18040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:54.498014 18040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:54.498059 18040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.498098 18040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:54.498113 18040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.507988 18040 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:36905
I20260929 18:40:54.508016 18168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:36905 every 8 connection(s)
I20260929 18:40:54.508922 18040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:54.509662 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18040
I20260929 18:40:54.509783 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:40:54.510049 11739 external_mini_cluster.cc:1428] Setting key ed3a11ab7cdef378716688a7a886882f
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.195@KRBTEST.COM" created.
I20260929 18:40:54.525578 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.510995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:56707 (local address 127.11.118.254:39579)
0929 18:40:54.511140 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:54.511145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:54.512293 (+ 1148us) server_negotiation.cc:407] Connection header received
0929 18:40:54.514346 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:54.514350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:54.514407 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:54.514487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:54.515696 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.516586 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.517760 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.518200 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.518364 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.521127 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.521130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.521132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.521158 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:54.522776 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.523566 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.523569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.523570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.523655 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.524124 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.524130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.524132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.524385 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:54.524475 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:54.525157 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 18:40:54.525321 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:40:54.526389 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.511595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:56707)
0929 18:40:54.512137 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:54.512205 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:54.514062 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:54.514668 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:54.514676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:54.515044 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:54.515547 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.515560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.516717 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.516720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.517625 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.517635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.518405 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.518408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.518555 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.519210 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:54.519228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:54.520912 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:54.522975 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.522982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.522997 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.523431 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.523806 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.523809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.523810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.523968 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.524513 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:54.524524 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:54.524893 (+ 369us) client_negotiation.cc:769] Sending connection context
0929 18:40:54.525068 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 18:40:54.525356 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":155,"threads_started":1}
I20260929 18:40:54.527594 18169 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:54.527859 18169 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:54.528458 18169 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
I20260929 18:40:54.529722 17844 ts_manager.cc:194] Registered new tserver with Master: 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
I20260929 18:40:54.530423 17844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:56707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
I20260929 18:40:54.542022 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:54.662611 18174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:54.662925 18174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:54.663009 18174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:54.667296 18174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:54.667384 18174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:54.667474 18174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:54.673142 18174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:54.674362 18174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:54.675417 18174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:54.685822 18179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.685815 18180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.685854 18182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:54.686019 18174 server_base.cc:1034] running on GCE node
I20260929 18:40:54.686594 18174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:54.687304 18174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:54.688517 18174 hybrid_clock.cc:648] HybridClock initialized: now 1790707254688502 us; error 38 us; skew 500 ppm
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:54.691632 18174 init.cc:377] Logged in from keytab as oryx/127.11.118.195@KRBTEST.COM (short username oryx)
I20260929 18:40:54.692595 18174 webserver.cc:460] Webserver started at http://127.11.118.195:42597/ using document root <none> and password file <none>
I20260929 18:40:54.693228 18174 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:54.693333 18174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:54.693573 18174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:54.695770 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "23eb9d8dfdcdbe684d20ebd0292bab01"
server_key_iv: "2b72618efb5602fa83b857a3d58ac8c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.696399 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "23eb9d8dfdcdbe684d20ebd0292bab01"
server_key_iv: "2b72618efb5602fa83b857a3d58ac8c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.700752 18174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:40:54.703507 18189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.704854 18174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:40:54.705001 18174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "23eb9d8dfdcdbe684d20ebd0292bab01"
server_key_iv: "2b72618efb5602fa83b857a3d58ac8c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.705124 18174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:54.722126 18174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:54.725632 18174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:54.725847 18174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:54.726518 18174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:54.727367 18174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:54.727437 18174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.727507 18174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:54.727568 18174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.737109 18174 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:46495
I20260929 18:40:54.737088 18302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:46495 every 8 connection(s)
I20260929 18:40:54.738164 18174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:54.739216 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18174
I20260929 18:40:54.739353 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:40:54.739610 11739 external_mini_cluster.cc:1428] Setting key 09c1b7a7d7e79442670ac1fa0301812b
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:40:54.752104 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.740234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:41953 (local address 127.11.118.254:39579)
0929 18:40:54.740377 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:54.740381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:54.741443 (+ 1062us) server_negotiation.cc:407] Connection header received
0929 18:40:54.742940 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:54.742943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:54.742986 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:54.743051 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:54.744137 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.744660 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.745794 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.746044 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.746125 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.748635 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.748638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.748640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.748664 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:54.750008 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.750543 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.750546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.750547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.750637 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:54.751019 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:54.751021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:54.751022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:54.751201 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:54.751287 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:54.751804 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 18:40:54.751945 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:40:54.752894 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.740731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:41953)
0929 18:40:54.741274 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:54.741332 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:54.742726 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:54.743184 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:54.743191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:54.743539 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:54.744020 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.744032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.744814 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.744817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.745681 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.745689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.746178 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.746181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.746403 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.747030 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:54.747046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:54.748486 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:54.750113 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.750118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.750129 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.750448 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.750796 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:54.750798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:54.750799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:54.750922 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:54.751287 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:54.751293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:54.751516 (+ 223us) client_negotiation.cc:769] Sending connection context
0929 18:40:54.751665 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 18:40:54.751936 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
I20260929 18:40:54.754179 18303 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:54.754454 18303 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:54.754980 18303 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
I20260929 18:40:54.755972 17844 ts_manager.cc:194] Registered new tserver with Master: 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:40:54.756520 17844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:41953
I20260929 18:40:54.765761 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.118.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.253.keytab.
Entry for principal oryx/127.11.118.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.253.keytab.
I20260929 18:40:54.793073 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:33103
--webserver_interface=127.11.118.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.253.keytab
--principal=oryx/127.11.118.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39579,127.11.118.253:33103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:40:54.905378 18308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:54.905742 18308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:54.905825 18308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:54.910305 18308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:54.910403 18308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:54.910441 18308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:54.910480 18308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:54.910519 18308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:54.916531 18308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39579,127.11.118.253:33103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.253.keytab
--principal=oryx/127.11.118.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:33103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.118.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
W20260929 18:40:54.916831 18308 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.118.254:39579,127.11.118.253:33103'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 18:40:54.917801 18308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:54.918910 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:54.927953 18314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:54.927953 18313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:54.928016 18308 server_base.cc:1034] running on GCE node
W20260929 18:40:54.927953 18316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:54.928826 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:54.930137 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:54.931305 18308 hybrid_clock.cc:648] HybridClock initialized: now 1790707254931284 us; error 38 us; skew 500 ppm
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:54.934183 18308 init.cc:377] Logged in from keytab as oryx/127.11.118.253@KRBTEST.COM (short username oryx)
I20260929 18:40:54.935204 18308 webserver.cc:460] Webserver started at http://127.11.118.253:43185/ using document root <none> and password file <none>
I20260929 18:40:54.935772 18308 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:54.935837 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:54.936007 18308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:54.938447 18308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/instance:
uuid: "5194fde6067d4b9c968d19dd3e639178"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "b22a42dc534920fbea8df956c18eceac"
server_key_iv: "0ccb54111f2a2e634d2ec382fdc1ef94"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.939105 18308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal/instance:
uuid: "5194fde6067d4b9c968d19dd3e639178"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "b22a42dc534920fbea8df956c18eceac"
server_key_iv: "0ccb54111f2a2e634d2ec382fdc1ef94"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.943154 18308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:40:54.945709 18323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:54.947098 18308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:40:54.947275 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "5194fde6067d4b9c968d19dd3e639178"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "b22a42dc534920fbea8df956c18eceac"
server_key_iv: "0ccb54111f2a2e634d2ec382fdc1ef94"
server_key_version: "encryptionkey@0"
I20260929 18:40:54.947402 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:54.968524 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:54.971781 18308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:54.972008 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:54.979463 18308 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.253:33103
I20260929 18:40:54.979760 18375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.253:33103 every 8 connection(s)
I20260929 18:40:54.980530 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
I20260929 18:40:54.983505 18376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:54.987438 18376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:40:54.989846 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18308
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.253@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
Sep 29 18:40:54 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:55.003963 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.989444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.11.118.254:39579)
0929 18:40:54.989602 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:54.989608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:54.990625 (+ 1017us) server_negotiation.cc:407] Connection header received
0929 18:40:54.992044 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:54.992048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:54.992106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:54.992217 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:54.993350 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.993877 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.995156 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.995486 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.995592 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.999589 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.999593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.999596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.999628 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.001685 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.002330 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.002333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.002335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.002428 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.002817 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.002820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.002822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.003049 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.003163 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.003660 (+ 497us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.003793 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:55.004921 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.990928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.11.118.253:33103)
0929 18:40:54.991234 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:54.991244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:54.991283 (+ 39us) server_negotiation.cc:407] Connection header received
0929 18:40:54.991849 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:54.991874 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:54.992113 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:54.992306 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:54.993239 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.994699 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.995680 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.996169 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.996306 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.999496 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:54.999503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:54.999538 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:54.999565 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.002009 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.002538 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.002544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.002546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.002699 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.003001 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.003004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.003006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.003414 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.003636 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.003886 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.004034 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":212,"spinlock_wait_cycles":18048,"thread_start_us":121,"threads_started":1}
I20260929 18:40:55.004928 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:54.989868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57706)
0929 18:40:54.990466 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:54.990528 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:54.991830 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:54.992266 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:54.992272 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:54.992812 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:54.993193 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.993215 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.994013 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.994016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.995026 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:54.995036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:54.995660 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:54.995666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:54.995883 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:54.996686 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:54.996712 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:54.999383 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.001842 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.001854 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.001864 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.002130 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.002534 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.002538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.002540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.002692 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.003159 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.003167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.003427 (+ 260us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.003604 (+ 177us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.003945 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":42,"spinlock_wait_cycles":1792,"thread_start_us":155,"threads_started":1}
I20260929 18:40:55.006388 18376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707254, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.253@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:55.015925 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.006921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56328)
0929 18:40:55.007076 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.007091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.007155 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.007438 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.007440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.007615 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.007695 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.007700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.008509 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.008512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.009025 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.009032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.009485 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.009488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.009581 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.010066 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.010078 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.011508 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.014252 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.014255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.014257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.014503 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.014786 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.014791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.014793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.014870 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.015293 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.015296 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.015382 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.015496 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.015697 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":4}
I20260929 18:40:55.015925 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.007009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.11.118.253:33103)
0929 18:40:55.007116 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.007119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.007141 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:40:55.007280 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.007282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.007322 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.007431 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.007838 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.008343 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.009139 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.009376 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.009485 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.011663 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.011666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.011668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.011687 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.014109 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.014606 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.014611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.014613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.014686 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.014960 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.014965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.014966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.015180 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.015303 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.015538 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.015756 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":42}
I20260929 18:40:55.019198 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Bootstrap starting.
I20260929 18:40:55.021497 18376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:55.022240 18376 log.cc:824] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:55.024240 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: No bootstrap required, opened a new log
I20260929 18:40:55.027240 18376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } }
I20260929 18:40:55.027449 18376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:55.027523 18376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5194fde6067d4b9c968d19dd3e639178, State: Initialized, Role: FOLLOWER
I20260929 18:40:55.027931 18376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } }
I20260929 18:40:55.028414 18382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } } }
I20260929 18:40:55.028573 18382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:40:55.029143 18376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 18:40:55.034063 18393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:55.034159 18393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:55.036078 18308 master_runner.cc:428] Detected that this master 5194fde6067d4b9c968d19dd3e639178 is joining an existing cluster
I20260929 18:40:55.036145 18308 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.118.253:33103
I20260929 18:40:55.046631 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.038812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.11.118.253:33103)
0929 18:40:55.038921 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.038926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.039389 (+ 463us) server_negotiation.cc:407] Connection header received
0929 18:40:55.039491 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.039497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.039538 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.039618 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.040034 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.040510 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.041335 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.041563 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.041695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.042993 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.042996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.042998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.043021 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.044827 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.045383 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.045388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.045390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.045467 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.045746 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.045749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.045751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.045967 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.046020 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.046318 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.046469 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":43}
I20260929 18:40:55.046643 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.038499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56342)
0929 18:40:55.039304 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.039315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.039421 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.039630 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.039632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.039829 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.039925 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.039930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.040614 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.040620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.041219 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.041226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.041705 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.041708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.041791 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.042348 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.042368 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.042851 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.044969 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.044972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.044974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.045268 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.045571 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.045573 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.045576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.045653 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.046063 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.046064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.046121 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.046245 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.046415 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"spinlock_wait_cycles":6144,"thread_start_us":90,"threads_started":1}
I20260929 18:40:55.047688 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.038858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57712 (local address 127.11.118.254:39579)
0929 18:40:55.038969 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.038974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.038986 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:40:55.039033 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.039035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.039071 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.039147 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.039721 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.040559 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.041787 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.042133 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.042210 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.043601 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.043607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.043612 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.043646 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.045782 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.046324 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.046327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.046329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.046419 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.046719 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.046722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.046726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.046972 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.047076 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.047409 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.047530 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 18:40:55.048188 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.038485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57712)
0929 18:40:55.038776 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.038792 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.038883 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.039194 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.039198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.039433 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.039579 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.039586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.040703 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.040707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.041644 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.041654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.042399 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.042402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.042492 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.042931 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.042945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.043400 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.045896 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.045899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.045902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.046156 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.046533 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.046536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.046538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.046636 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.047084 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.047087 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.047192 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.047878 (+ 686us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.048021 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"spinlock_wait_cycles":768,"thread_start_us":99,"threads_started":1}
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.254@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:55.063272 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.052936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56350)
0929 18:40:55.053372 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.053430 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.053574 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.053879 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.053885 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.054387 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.054529 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.054538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.055255 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.055258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.056264 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.056275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.056717 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.056721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.056834 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.057375 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.057390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.059144 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.061246 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.061256 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.061263 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.061581 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.061910 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.061913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.061915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.062010 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.062567 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.062574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.062715 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.062875 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.063078 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":122,"threads_started":1}
I20260929 18:40:55.063336 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.052965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.11.118.253:33103)
0929 18:40:55.053107 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.053111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.053520 (+ 409us) server_negotiation.cc:407] Connection header received
0929 18:40:55.053708 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.053710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.053756 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.053825 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.054645 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.055149 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.056403 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.056611 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.056688 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.059330 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.059334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.059337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.059365 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.061119 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.061690 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.061697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.061699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.061803 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.062115 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.062119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.062123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.062450 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.062561 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.062956 (+ 395us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.063097 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:40:55.065430 17844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.118.253:33103: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5194fde6067d4b9c968d19dd3e639178"
member_type: NON_VOTER
last_known_addr {
host: "127.11.118.253"
port: 33103
}
attrs {
promote: true
}
}
dest_uuid: "71f51d3eac064c56a2d30ff52c498988"
cas_config_opid_index: -1
I20260929 18:40:55.066256 17844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.11.118.252@KRBTEST.COM; defaulting to no policy
I20260929 18:40:55.069087 18404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } }
I20260929 18:40:55.069566 18350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:55.071673 18404 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.118.253:33103
I20260929 18:40:55.071908 18350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 18:40:55.071924 18404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } }
I20260929 18:40:55.072013 18404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:55.072571 18308 master.cc:537] Master@127.11.118.253:33103 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.252@KRBTEST.COM" created.
I20260929 18:40:55.073026 18405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:40:55.075534 17831 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260929 18:40:55.077220 18308 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:55.077625 18308 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:55.077836 18308 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: stopping tablet replica
I20260929 18:40:55.083173 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.082852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56360)
0929 18:40:55.083002 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.083075 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 18:40:55.083357 17831 consensus_peers.cc:487] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 -> Peer 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253:33103): Couldn't send request to peer 5194fde6067d4b9c968d19dd3e639178. Status: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.252.keytab.
Entry for principal oryx/127.11.118.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.252.keytab.
I20260929 18:40:55.093513 18308 master.cc:552] Master@127.11.118.253:33103 shutdown complete.
I20260929 18:40:55.093580 18308 master_runner.cc:305] Clearing existing system tablet
I20260929 18:40:55.093762 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:43355
--webserver_interface=127.11.118.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.252.keytab
--principal=oryx/127.11.118.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:40:55.093829 18308 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.093874 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.095178 18308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.096340 18411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.096616 18308 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.096793 18308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.096833 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "5194fde6067d4b9c968d19dd3e639178"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "b22a42dc534920fbea8df956c18eceac"
server_key_iv: "0ccb54111f2a2e634d2ec382fdc1ef94"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.099660 18308 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:55.102842 18308 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 18:40:55.102960 18308 log.cc:1196] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 18:40:55.103260 18308 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Deleting consensus metadata
I20260929 18:40:55.103458 18308 master_runner.cc:315] Copying system tablet from 127.11.118.254:39579
I20260929 18:40:55.104943 18308 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: tablet copy: Beginning tablet copy session from remote peer at address 127.11.118.254:39579
I20260929 18:40:55.114696 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.105722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57716 (local address 127.11.118.254:39579)
0929 18:40:55.105879 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.105884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.106005 (+ 121us) server_negotiation.cc:407] Connection header received
0929 18:40:55.106131 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.106135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.106186 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.106312 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.106858 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.107690 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.108974 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.109325 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.109396 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.110897 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.110900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.110903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.110929 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.112669 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.113243 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.113248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.113251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.113344 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.113712 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.113715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.113717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.113954 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.114022 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.114402 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.114524 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 18:40:55.115293 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.105580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57716)
0929 18:40:55.105901 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.105917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.106034 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.106382 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.106386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.106595 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.106732 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.106740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.107833 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.107836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.108841 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.108852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.109553 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.109557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.109664 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.110261 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.110296 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.110739 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.112834 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.112838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.112840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.113127 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.113476 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.113482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.113486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.113588 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.114092 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.114095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.114188 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.114936 (+ 748us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.115095 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":91,"threads_started":1}
I20260929 18:40:55.115972 17864 tablet_copy_service.cc:140] P 71f51d3eac064c56a2d30ff52c498988: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5194fde6067d4b9c968d19dd3e639178 ({username='oryx', principal='oryx/127.11.118.253@KRBTEST.COM'} at 127.0.0.1:57716)
I20260929 18:40:55.116169 17864 tablet_copy_service.cc:161] P 71f51d3eac064c56a2d30ff52c498988: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5194fde6067d4b9c968d19dd3e639178 at {username='oryx', principal='oryx/127.11.118.253@KRBTEST.COM'} at 127.0.0.1:57716: session id = 5194fde6067d4b9c968d19dd3e639178-00000000000000000000000000000000
I20260929 18:40:55.118012 17864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 18:40:55.119913 18308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 18:40:55.122514 18308 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: tablet copy: Starting download of 0 data blocks...
I20260929 18:40:55.122817 18308 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: tablet copy: Starting download of 1 WAL segments...
I20260929 18:40:55.125126 18308 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 18:40:55.127318 17864 tablet_copy_service.cc:342] P 71f51d3eac064c56a2d30ff52c498988: Request end of tablet copy session 5194fde6067d4b9c968d19dd3e639178-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.118.253@KRBTEST.COM'} at 127.0.0.1:57716
I20260929 18:40:55.127471 17864 tablet_copy_service.cc:433] P 71f51d3eac064c56a2d30ff52c498988: ending tablet copy session 5194fde6067d4b9c968d19dd3e639178-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5194fde6067d4b9c968d19dd3e639178
I20260929 18:40:55.135485 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:55.137365 18418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.137422 18419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.137665 18421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:55.138302 18308 server_base.cc:1034] running on GCE node
I20260929 18:40:55.138468 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:55.138777 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:55.139927 18308 hybrid_clock.cc:648] HybridClock initialized: now 1790707255139913 us; error 30 us; skew 500 ppm
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:55.141659 18308 init.cc:377] Logged in from keytab as oryx/127.11.118.253@KRBTEST.COM (short username oryx)
I20260929 18:40:55.142207 18308 webserver.cc:460] Webserver started at http://127.11.118.253:36271/ using document root <none> and password file <none>
I20260929 18:40:55.142429 18308 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.142479 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.143855 18308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.144793 18428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.145083 18308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.145156 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "5194fde6067d4b9c968d19dd3e639178"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "b22a42dc534920fbea8df956c18eceac"
server_key_iv: "0ccb54111f2a2e634d2ec382fdc1ef94"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.145228 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.165784 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:55.168077 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:55.172695 18308 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.253:33103
I20260929 18:40:55.172737 18480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.253:33103 every 8 connection(s)
I20260929 18:40:55.173273 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
I20260929 18:40:55.175079 18481 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:40:55.176164 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Bootstrap starting.
I20260929 18:40:55.186200 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:40:55.186542 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: Bootstrap complete.
I20260929 18:40:55.187130 18481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } }
I20260929 18:40:55.187264 18481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5194fde6067d4b9c968d19dd3e639178, State: Initialized, Role: LEARNER
I20260929 18:40:55.187371 18481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } }
I20260929 18:40:55.187703 18484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } }
I20260929 18:40:55.187804 18481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:55.187923 18484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: LEARNER
I20260929 18:40:55.191454 18495 catalog_manager.cc:1259] Loaded cluster ID: 7baea67fa1604e70b6ce11601db023ab
I20260929 18:40:55.191519 18495 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: loading cluster ID for follower catalog manager: success
I20260929 18:40:55.193622 18495 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: acquiring CA information for follower catalog manager: success
I20260929 18:40:55.195122 18495 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 18:40:55.232926 18409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:55.233201 18409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:55.233282 18409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:55.237650 18409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:40:55.237715 18409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:55.237737 18409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:55.237785 18409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:40:55.237813 18409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:40:55.243435 18409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.252.keytab
--principal=oryx/127.11.118.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.118.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:55.244601 18409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:55.245714 18409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:55.254256 18500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.254308 18503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.254245 18501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:55.254639 18409 server_base.cc:1034] running on GCE node
I20260929 18:40:55.255227 18409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:55.256436 18409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:55.257619 18409 hybrid_clock.cc:648] HybridClock initialized: now 1790707255257606 us; error 35 us; skew 500 ppm
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:55.260494 18409 init.cc:377] Logged in from keytab as oryx/127.11.118.252@KRBTEST.COM (short username oryx)
I20260929 18:40:55.261463 18409 webserver.cc:460] Webserver started at http://127.11.118.252:45703/ using document root <none> and password file <none>
I20260929 18:40:55.262068 18409 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.262159 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.262447 18409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:40:55.264624 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/instance:
uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
format_stamp: "Formatted at 2026-09-29 18:40:55 on dist-test-slave-cgv7"
server_key: "2377cb7fb96c11112876a0c8c006c5c7"
server_key_iv: "33dd1d579bacec0cf0bc86039fea8c4e"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.265231 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal/instance:
uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
format_stamp: "Formatted at 2026-09-29 18:40:55 on dist-test-slave-cgv7"
server_key: "2377cb7fb96c11112876a0c8c006c5c7"
server_key_iv: "33dd1d579bacec0cf0bc86039fea8c4e"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.269311 18409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:40:55.272101 18510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.273576 18409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:55.273768 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
format_stamp: "Formatted at 2026-09-29 18:40:55 on dist-test-slave-cgv7"
server_key: "2377cb7fb96c11112876a0c8c006c5c7"
server_key_iv: "33dd1d579bacec0cf0bc86039fea8c4e"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.273902 18409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.289219 18409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:55.292500 18409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:55.292802 18409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:55.300153 18409 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.252:43355
I20260929 18:40:55.300447 18562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.252:43355 every 8 connection(s)
I20260929 18:40:55.301236 18409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
I20260929 18:40:55.304112 18563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:55.308038 18563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:40:55.310775 18035 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
I20260929 18:40:55.311041 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18409
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.252@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
I20260929 18:40:55.322412 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.310328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.11.118.254:39579)
0929 18:40:55.310466 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.310469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.311537 (+ 1068us) server_negotiation.cc:407] Connection header received
0929 18:40:55.312927 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.312929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.312974 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.313033 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.314044 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.314639 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.315784 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.316004 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.316117 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.318686 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.318690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.318692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.318717 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.320254 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.320798 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.320806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.320807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.320879 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.321226 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.321230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.321231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.321425 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.321531 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.322062 (+ 531us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.322185 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:40:55.323329 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.311922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.11.118.252:43355)
0929 18:40:55.312435 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.312446 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.312488 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:40:55.312697 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.312787 (+ 90us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.313053 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.313234 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.314031 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.315109 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.315948 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.316324 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.316443 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.319082 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.319092 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.319109 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.319138 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.321050 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.321469 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.321475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.321477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.321557 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.321833 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.321837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.321839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.322083 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.322305 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.322817 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.322931 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":149,"threads_started":1}
I20260929 18:40:55.323326 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.310542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57724)
0929 18:40:55.311368 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.311431 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.312697 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.313105 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.313116 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.313572 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.313906 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.313920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.314770 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.314773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.315654 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.315662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.316159 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.316163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.316337 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.316956 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.316980 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.318541 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.320360 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.320367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.320377 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.320695 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.320966 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.320969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.320971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.321147 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.321544 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.321552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.321820 (+ 268us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.322020 (+ 200us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.322358 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":31,"thread_start_us":185,"threads_started":1}
I20260929 18:40:55.324677 18563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.252@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:55.333134 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.325160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56374)
0929 18:40:55.325587 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.325601 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.325670 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.325922 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.325925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.326061 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.326156 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.326162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.327123 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.327126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.327690 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.327699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.328125 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.328128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.328207 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.328642 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.328653 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.329910 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.331696 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.331698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.331700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.331955 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.332218 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.332220 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.332221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.332303 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.332659 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.332661 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.332747 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.332870 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.333006 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354}
I20260929 18:40:55.333168 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.325316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.11.118.253:33103)
0929 18:40:55.325635 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.325640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.325655 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:55.325736 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.325738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.325778 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.325860 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.326302 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.327006 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.327809 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.328027 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.328101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.330041 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.330044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.330047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.330070 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.331580 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.332052 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.332055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.332056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.332137 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.332381 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.332383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.332384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.332564 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.332650 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.332939 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.333046 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260929 18:40:55.334034 18563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.252@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
I20260929 18:40:55.341846 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.334611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:40972)
0929 18:40:55.334754 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.334770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.334899 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.335137 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.335140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.335306 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.335369 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.335374 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.336104 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.336107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.336603 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.336609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.337132 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.337135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.337214 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.337757 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.337769 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.339064 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.340673 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.340675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.340677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.340873 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.341096 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.341098 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.341099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.341167 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.341475 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.341477 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.341550 (+ 73us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.341628 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.341720 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 18:40:55.341936 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.334644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.11.118.252:43355)
0929 18:40:55.334756 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.334761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.334856 (+ 95us) server_negotiation.cc:407] Connection header received
0929 18:40:55.334991 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.334994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.335031 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.335116 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.335472 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.335986 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.336709 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.336988 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.337105 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.339183 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.339186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.339188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.339207 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.340578 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.340954 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.340956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.340957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.341023 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.341241 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.341243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.341245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.341400 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.341471 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.341680 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.341805 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 18:40:55.345435 18563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Bootstrap starting.
I20260929 18:40:55.347819 18563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:55.348548 18563 log.cc:824] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:55.350517 18563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: No bootstrap required, opened a new log
I20260929 18:40:55.353498 18563 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } }
I20260929 18:40:55.353734 18563 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:55.353820 18563 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3d239fdefa4ba89ec98265eedf0a0f, State: Initialized, Role: FOLLOWER
I20260929 18:40:55.354338 18563 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } }
I20260929 18:40:55.354790 18569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } } }
I20260929 18:40:55.354916 18569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:40:55.355535 18563 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: configured and running, proceeding with master startup.
W20260929 18:40:55.360275 18580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:40:55.360376 18580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:40:55.364789 18409 master_runner.cc:428] Detected that this master fa3d239fdefa4ba89ec98265eedf0a0f is joining an existing cluster
I20260929 18:40:55.364871 18409 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.118.252:43355
I20260929 18:40:55.376019 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.367441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.11.118.254:39579)
0929 18:40:55.367566 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.367570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.367962 (+ 392us) server_negotiation.cc:407] Connection header received
0929 18:40:55.368153 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.368157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.368218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.368287 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.368968 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.369764 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371053 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.371410 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371494 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.372862 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.372865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.372868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.372898 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.374214 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.374793 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.374797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.374799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.374879 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.375161 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.375165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.375167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.375360 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.375468 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.375748 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.375863 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 18:40:55.376046 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.367367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57736)
0929 18:40:55.367877 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.367891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.368034 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.368450 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.368453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.368686 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.368834 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.368841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.369897 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.369900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.370924 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.370933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.371665 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.371757 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.372334 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.372344 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.372715 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.374485 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.374488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.374491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.374688 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.374981 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.374984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.374987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.375069 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.375467 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.375470 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.375575 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.375680 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.375838 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":255,"spinlock_wait_cycles":1408,"thread_start_us":84,"threads_started":1}
I20260929 18:40:55.376893 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17906
I20260929 18:40:55.384232 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:42325
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:39579
--webserver_port=45525
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707253, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.254@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
I20260929 18:40:55.390756 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.380702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:40990)
0929 18:40:55.380846 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.380862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.380943 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.381238 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.381241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.381436 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.381519 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.381524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.382363 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.382368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.383011 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.383021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.383539 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.383543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.383637 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.384189 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.384204 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.385859 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.388774 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.388778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.388780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.389075 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.389509 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.389513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.389515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.389612 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.390103 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.390108 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.390242 (+ 134us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.390363 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.390535 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 18:40:55.391518 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.380749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.11.118.252:43355)
0929 18:40:55.380986 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.380992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.381004 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:40:55.381058 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.381060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.381108 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.381228 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.381630 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.382213 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.383125 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.383389 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.383900 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.386016 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.386020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.386022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.386054 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.388638 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.389295 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.389300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.389303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.389393 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.389736 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.389742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.389745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.389988 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.391092 (+ 1104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.391178 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.391308 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260929 18:40:55.392410 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.368238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56388)
0929 18:40:55.368575 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.368588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.368737 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.369115 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.369118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.369332 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.369433 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.369440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.370518 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.370521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.371382 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.371390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.372112 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.372114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.372180 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.386934 (+ 14754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.386947 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.387913 (+ 966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.390331 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.390336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.390341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.390561 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.391370 (+ 809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.391373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.391375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.391453 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.391902 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.391905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.392003 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.392098 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.392250 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":28,"spinlock_wait_cycles":3328,"thread_start_us":97,"threads_started":1}
I20260929 18:40:55.392839 17844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.118.252:43355: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
member_type: NON_VOTER
last_known_addr {
host: "127.11.118.252"
port: 43355
}
attrs {
promote: true
}
}
dest_uuid: "71f51d3eac064c56a2d30ff52c498988"
cas_config_opid_index: 5
I20260929 18:40:55.393000 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.368002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.11.118.253:33103)
0929 18:40:55.368131 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.368135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.368658 (+ 523us) server_negotiation.cc:407] Connection header received
0929 18:40:55.368855 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.368859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.368919 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.368975 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.369563 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.370381 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371514 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.371872 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371961 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.388107 (+ 16146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.388111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.388114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.388150 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.389903 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.391161 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.391165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.391167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.391261 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.391559 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.391562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.391566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.391793 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.391912 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.392707 (+ 795us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.392846 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:40:55.393510 17844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } }
I20260929 18:40:55.394729 18405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.396328 18405 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.118.252:43355
I20260929 18:40:55.396512 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.396610 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:55.397459 18409 master.cc:537] Master@127.11.118.252:43355 shutting down...
I20260929 18:40:55.397658 18537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:55.399513 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.367367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:40974)
0929 18:40:55.367693 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.367709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.367851 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.368164 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.368166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.368402 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.368538 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.368546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.369546 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.369549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.371019 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.371029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.371682 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.371804 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.386759 (+ 14955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.386771 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.387321 (+ 550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.397919 (+ 10598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.397923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.397925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.398187 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.398496 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.398500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.398504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.398589 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.399014 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.399016 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.399111 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.399197 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.399339 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":50,"thread_start_us":121,"threads_started":1}
I20260929 18:40:55.399622 18537 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 18:40:55.400055 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.367430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.11.118.252:43355)
0929 18:40:55.367551 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.367556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.367774 (+ 218us) server_negotiation.cc:407] Connection header received
0929 18:40:55.367947 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.367951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.367988 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.368033 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.368656 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.369422 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371141 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.371487 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.371552 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.387481 (+ 15929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.387486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.387492 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.387517 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.391041 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.398316 (+ 7275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.398319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.398321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.398402 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.398691 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.398694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.398695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.398885 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.398981 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.399788 (+ 807us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.399895 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":43}
I20260929 18:40:55.400498 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.394858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56400)
0929 18:40:55.395283 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.395298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.395372 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.395819 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.395823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.395841 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:55.395930 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.395936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.397113 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.397117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.398378 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.398388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.399818 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.399825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.399919 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.399990 (+ 71us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.400072 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.400399 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350}
I20260929 18:40:55.401007 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.395070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.11.118.253:33103)
0929 18:40:55.395508 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.395512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.395523 (+ 11us) server_negotiation.cc:407] Connection header received
0929 18:40:55.395559 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.395562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.395613 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.395700 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:55.396052 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.396991 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.398560 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.399242 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.399349 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.399494 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.400782 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.400874 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260929 18:40:55.401506 18455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 LEARNER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 18:40:55.401681 18404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:55.401998 18405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260929 18:40:55.403544 17828 consensus_peers.cc:487] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 -> Peer fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252:43355): Couldn't send request to peer fa3d239fdefa4ba89ec98265eedf0a0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 18:40:55.404107 18455 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.405771 18592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f51d3eac064c56a2d30ff52c498988. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.405891 18592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: LEARNER
I20260929 18:40:55.405891 18484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: NON_VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: true } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.405997 18484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: LEARNER
I20260929 18:40:55.406572 18404 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: attempting to promote NON_VOTER 5194fde6067d4b9c968d19dd3e639178 to VOTER
I20260929 18:40:55.407053 18404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } }
W20260929 18:40:55.408854 17828 consensus_peers.cc:487] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 -> Peer fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252:43355): Couldn't send request to peer fa3d239fdefa4ba89ec98265eedf0a0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 18:40:55.408988 18454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 LEARNER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 18:40:55.409399 18404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 18:40:55.409976 18409 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:55.410311 18409 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:55.410429 18409 tablet_replica.cc:330] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: stopping tablet replica
I20260929 18:40:55.412177 18405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.413964 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.414075 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:55.414973 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.414642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:41000)
0929 18:40:55.414819 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.414887 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260929 18:40:55.415369 18404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.415480 18404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:55.416217 18454 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.417627 18484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.417748 18484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:40:55.426807 18409 master.cc:552] Master@127.11.118.252:43355 shutdown complete.
I20260929 18:40:55.426884 18409 master_runner.cc:305] Clearing existing system tablet
I20260929 18:40:55.427119 18409 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.427191 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.428320 18409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 18:40:55.429178 18596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.429404 18409 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.429533 18409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.429566 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
format_stamp: "Formatted at 2026-09-29 18:40:55 on dist-test-slave-cgv7"
server_key: "2377cb7fb96c11112876a0c8c006c5c7"
server_key_iv: "33dd1d579bacec0cf0bc86039fea8c4e"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.431403 18409 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:55.433326 18409 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 18:40:55.433437 18409 log.cc:1196] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 18:40:55.433668 18409 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Deleting consensus metadata
I20260929 18:40:55.433943 18409 master_runner.cc:315] Copying system tablet from 127.11.118.254:39579
I20260929 18:40:55.435174 18409 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: tablet copy: Beginning tablet copy session from remote peer at address 127.11.118.254:39579
I20260929 18:40:55.443032 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.435825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.11.118.254:39579)
0929 18:40:55.435965 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.435970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.436146 (+ 176us) server_negotiation.cc:407] Connection header received
0929 18:40:55.436315 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.436319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.436371 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.436458 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.436906 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.437436 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.438329 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.438560 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.438676 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.439810 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.439813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.439816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.439842 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.441251 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.441749 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.441752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.441755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.441833 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.442114 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.442117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.442118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.442353 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.442459 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.442763 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.442871 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:40:55.443086 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.435738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:57740)
0929 18:40:55.436052 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.436064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.436196 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.436476 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.436479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.436690 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.436788 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.436794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.437545 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.437551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.438133 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.438141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.438679 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.438682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.438757 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.439337 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.439350 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.439667 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.441379 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.441383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.441385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.441650 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.441927 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.441932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.441934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.442019 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.442459 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.442463 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.442565 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.442685 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.442826 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":88,"threads_started":1}
I20260929 18:40:55.443765 17864 tablet_copy_service.cc:140] P 71f51d3eac064c56a2d30ff52c498988: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fa3d239fdefa4ba89ec98265eedf0a0f ({username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:57740)
I20260929 18:40:55.443885 17864 tablet_copy_service.cc:161] P 71f51d3eac064c56a2d30ff52c498988: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fa3d239fdefa4ba89ec98265eedf0a0f at {username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:57740: session id = fa3d239fdefa4ba89ec98265eedf0a0f-00000000000000000000000000000000
I20260929 18:40:55.444770 17864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 18:40:55.446026 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 18:40:55.447988 18409 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: tablet copy: Starting download of 0 data blocks...
I20260929 18:40:55.448189 18409 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: tablet copy: Starting download of 1 WAL segments...
I20260929 18:40:55.449632 18409 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 18:40:55.451421 17864 tablet_copy_service.cc:342] P 71f51d3eac064c56a2d30ff52c498988: Request end of tablet copy session fa3d239fdefa4ba89ec98265eedf0a0f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:57740
I20260929 18:40:55.451526 17864 tablet_copy_service.cc:433] P 71f51d3eac064c56a2d30ff52c498988: ending tablet copy session fa3d239fdefa4ba89ec98265eedf0a0f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fa3d239fdefa4ba89ec98265eedf0a0f
I20260929 18:40:55.458506 18409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:55.460330 18603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.467547 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.467547 18604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:55.467968 18409 server_base.cc:1034] running on GCE node
I20260929 18:40:55.468163 18409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:55.468519 18409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:55.469655 18409 hybrid_clock.cc:648] HybridClock initialized: now 1790707255469659 us; error 53 us; skew 500 ppm
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:55.471154 18409 init.cc:377] Logged in from keytab as oryx/127.11.118.252@KRBTEST.COM (short username oryx)
I20260929 18:40:55.471655 18409 webserver.cc:460] Webserver started at http://127.11.118.252:33963/ using document root <none> and password file <none>
I20260929 18:40:55.471872 18409 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.472002 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.473155 18409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.474107 18613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.474478 18409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 18:40:55.474581 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
format_stamp: "Formatted at 2026-09-29 18:40:55 on dist-test-slave-cgv7"
server_key: "2377cb7fb96c11112876a0c8c006c5c7"
server_key_iv: "33dd1d579bacec0cf0bc86039fea8c4e"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.474686 18409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.484743 18409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:55.486819 18409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:55.491820 18665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.252:43355 every 8 connection(s)
I20260929 18:40:55.491817 18409 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.252:43355
I20260929 18:40:55.492617 18409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
I20260929 18:40:55.499588 18666 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:40:55.500536 18666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Bootstrap starting.
I20260929 18:40:55.510087 18666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:40:55.510457 18666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Bootstrap complete.
I20260929 18:40:55.511130 18666 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } }
I20260929 18:40:55.511282 18666 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fa3d239fdefa4ba89ec98265eedf0a0f, State: Initialized, Role: LEARNER
I20260929 18:40:55.511392 18666 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } }
I20260929 18:40:55.511865 18666 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:40:55.513556 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.513988 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: LEARNER
I20260929 18:40:55.515774 18680 catalog_manager.cc:1259] Loaded cluster ID: 7baea67fa1604e70b6ce11601db023ab
I20260929 18:40:55.515851 18680 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: loading cluster ID for follower catalog manager: success
I20260929 18:40:55.517330 18680 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: acquiring CA information for follower catalog manager: success
I20260929 18:40:55.518560 18680 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:40:55.533221 18169 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
W20260929 18:40:55.555133 18590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:55.555505 18590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:55.555569 18590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:55.561887 18590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:55.561985 18590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:55.562078 18590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:40:55.569097 18590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:42325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=45525
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:55.570315 18590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:55.571406 18590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:55.581336 18688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.581305 18685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.581300 18686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:55.581526 18590 server_base.cc:1034] running on GCE node
I20260929 18:40:55.582190 18590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:55.582942 18590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:55.584129 18590 hybrid_clock.cc:648] HybridClock initialized: now 1790707255584113 us; error 27 us; skew 500 ppm
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:55.586934 18590 init.cc:377] Logged in from keytab as oryx/127.11.118.193@KRBTEST.COM (short username oryx)
I20260929 18:40:55.587882 18590 webserver.cc:460] Webserver started at http://127.11.118.193:45525/ using document root <none> and password file <none>
I20260929 18:40:55.588459 18590 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.588580 18590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.592525 18590 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:55.594722 18695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.596015 18590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:55.596144 18590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "f54790de3458470ba4da284f400e5bc8"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "2766a2f6068487d434048fc25aae7f61"
server_key_iv: "89600c5c954a7f55a08953fb5338c160"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.596695 18590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.609398 18590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:55.612895 18590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:55.613108 18590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:55.613826 18590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:55.614754 18590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:55.614801 18590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.614868 18590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:55.614900 18590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.623224 18590 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:42325
I20260929 18:40:55.623289 18808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:42325 every 8 connection(s)
I20260929 18:40:55.624323 18590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:40:55.634575 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18590
I20260929 18:40:55.634826 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18040
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:55.642545 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:44961 (local address 127.11.118.252:43355)
0929 18:40:55.626870 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.626884 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.627675 (+ 791us) server_negotiation.cc:407] Connection header received
0929 18:40:55.628958 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.628963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.629026 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.629113 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.630457 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631560 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632624 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.632950 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.633055 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.637755 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.637762 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.637768 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.637805 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.639900 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640545 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.640549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.640551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.640765 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.641067 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.641070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.641072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.641287 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.641373 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.642203 (+ 830us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.642366 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":74,"threads_started":1}
I20260929 18:40:55.642544 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:37053 (local address 127.11.118.254:39579)
0929 18:40:55.626631 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.626636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.627473 (+ 837us) server_negotiation.cc:407] Connection header received
0929 18:40:55.629068 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.629074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.629130 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.629182 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.630493 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631297 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632381 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.632710 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632825 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.637064 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.637068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.637071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.637102 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.639828 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640545 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.640549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.640551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.640651 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.641067 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.641070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.641072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.641298 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.641373 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.642203 (+ 830us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.642365 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260929 18:40:55.642670 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36395 (local address 127.11.118.253:33103)
0929 18:40:55.626721 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.626726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.627920 (+ 1194us) server_negotiation.cc:407] Connection header received
0929 18:40:55.628756 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.628759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.628845 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.628952 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.630478 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631017 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632439 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.632804 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632894 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.637751 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.637755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.637758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.637798 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.639614 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640482 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.640489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.640494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.640587 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.641024 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.641030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.641035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.641313 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.641398 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.642235 (+ 837us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.642425 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260929 18:40:55.643782 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:36905
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:39579
--webserver_port=38729
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:40:55.644462 18814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.11.118.193:44961)
0929 18:40:55.627601 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.627613 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.628830 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.629178 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.629187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.629743 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.630296 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.630317 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631672 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.631676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.632514 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.632521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.633060 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.633063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.633209 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.633928 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.633956 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.637302 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.640012 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.640023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640425 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.640860 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.640865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640958 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.641446 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.641453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.641886 (+ 433us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.643312 (+ 1426us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.643431 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":282,"spinlock_wait_cycles":93312,"thread_start_us":112,"threads_started":1}
I20260929 18:40:55.644467 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:37053)
0929 18:40:55.627315 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.627365 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.628950 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.629326 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.629328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.629698 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.630324 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.630337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631430 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.631434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.632248 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.632257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.632871 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.633074 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.633936 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.633966 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.636868 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.639965 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.639970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.639972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640425 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.640893 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.640902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640967 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.641446 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.641453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.641886 (+ 433us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.642866 (+ 980us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.643266 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":150,"spinlock_wait_cycles":28032,"thread_start_us":158,"threads_started":1}
I20260929 18:40:55.644740 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.626782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.11.118.193:36395)
0929 18:40:55.627823 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.627833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.628581 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.629178 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.629187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.629712 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.630305 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.630317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.631163 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.631167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.632296 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.632306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.632983 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.632987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.633093 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.633928 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.633956 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.637605 (+ 3649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.639764 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.639773 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.639794 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640360 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.640697 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.640703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.640707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.640908 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.641461 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.641467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.641886 (+ 419us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.642824 (+ 938us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.643640 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":872,"spinlock_wait_cycles":53504,"thread_start_us":96,"threads_started":1}
I20260929 18:40:55.646263 18809 heartbeater.cc:347] Connected to a master server at 127.11.118.252:43355
I20260929 18:40:55.646296 18810 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:55.646592 18809 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.646592 18810 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.647212 18811 heartbeater.cc:347] Connected to a master server at 127.11.118.253:33103
I20260929 18:40:55.647226 18809 heartbeater.cc:511] Master 127.11.118.252:43355 requested a full tablet report, sending...
I20260929 18:40:55.647318 18811 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.647228 18810 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
I20260929 18:40:55.647472 18811 heartbeater.cc:511] Master 127.11.118.253:33103 requested a full tablet report, sending...
I20260929 18:40:55.648752 18445 ts_manager.cc:194] Registered new tserver with Master: f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325)
I20260929 18:40:55.648993 18624 ts_manager.cc:194] Registered new tserver with Master: f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325)
I20260929 18:40:55.649541 17844 ts_manager.cc:194] Re-registered known tserver with Master: f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325)
I20260929 18:40:55.650152 17844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:37053
I20260929 18:40:55.758478 18303 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
W20260929 18:40:55.763289 18817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:55.763545 18817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:55.763587 18817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:55.767572 18817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:55.767685 18817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:55.767791 18817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:40:55.773310 18817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:36905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=38729
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:55.774552 18817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:55.775691 18817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:55.785563 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.785583 18822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:55.785563 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:55.785637 18817 server_base.cc:1034] running on GCE node
I20260929 18:40:55.786180 18817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:55.786933 18817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:55.788172 18817 hybrid_clock.cc:648] HybridClock initialized: now 1790707255788154 us; error 35 us; skew 500 ppm
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:55.791085 18817 init.cc:377] Logged in from keytab as oryx/127.11.118.194@KRBTEST.COM (short username oryx)
I20260929 18:40:55.791977 18817 webserver.cc:460] Webserver started at http://127.11.118.194:38729/ using document root <none> and password file <none>
I20260929 18:40:55.792495 18817 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:55.792564 18817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:55.796345 18817 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:40:55.798257 18832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.799527 18817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:40:55.799618 18817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "71bdf123505e4677a00bdf8b2e1e2201"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "c7103b8156f4d9525b4ca28d82aca205"
server_key_iv: "9d9db806c136403fbc5f6fd03be77802"
server_key_version: "encryptionkey@0"
I20260929 18:40:55.800208 18817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:55.832472 18817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:55.835776 18817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:55.835937 18817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:55.836491 18817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:55.837314 18817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:55.837360 18817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.837400 18817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:55.837415 18817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:55.846735 18817 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:36905
I20260929 18:40:55.846783 18945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:36905 every 8 connection(s)
I20260929 18:40:55.847757 18817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:40:55.852430 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18817
I20260929 18:40:55.852656 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18174
I20260929 18:40:55.859854 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:46495
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:39579
--webserver_port=42597
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
Sep 29 18:40:55 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
I20260929 18:40:55.867703 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.849585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:52591 (local address 127.11.118.252:43355)
0929 18:40:55.849723 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.849728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.851378 (+ 1650us) server_negotiation.cc:407] Connection header received
0929 18:40:55.851942 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.851946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.852006 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.852084 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.855117 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.855863 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.857400 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.857749 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.857826 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.862703 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.862707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.862710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.862742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.865017 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865639 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.865643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.865644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.865720 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.866173 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.866176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.866178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.866438 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.866523 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.867386 (+ 863us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.867526 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:40:55.867703 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.850211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:49819 (local address 127.11.118.254:39579)
0929 18:40:55.850357 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.850361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.851169 (+ 808us) server_negotiation.cc:407] Connection header received
0929 18:40:55.851970 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.851973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.852027 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.852080 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.854651 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.855520 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.857400 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.857743 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.857830 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.862703 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.862707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.862710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.862742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.864751 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865613 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.865619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.865621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.865712 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.866173 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.866176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.866178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.866438 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.866523 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.867386 (+ 863us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.867524 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 18:40:55.867774 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.850217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:42633 (local address 127.11.118.253:33103)
0929 18:40:55.850357 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.850361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.850804 (+ 443us) server_negotiation.cc:407] Connection header received
0929 18:40:55.851968 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.851972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.852027 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.852080 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:55.855122 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.856151 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.857700 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.858144 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.858253 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.862581 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:55.862589 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:55.862593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:55.862628 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:55.864946 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865697 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.865704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.865708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.865805 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:55.866221 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:55.866226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:55.866231 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:55.866541 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:55.866638 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.867398 (+ 760us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.867550 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 18:40:55.869819 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.850010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.11.118.194:52591)
0929 18:40:55.851293 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.851305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.851752 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.852272 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.852289 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.854765 (+ 2476us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.854960 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.854968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.856000 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.856003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.857254 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.857265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.858052 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.858056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.858417 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.859393 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.859410 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.862520 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.865134 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.865139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.865529 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.865840 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.865845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.866059 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.866594 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.866602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.867062 (+ 460us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.868095 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.868676 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":870,"spinlock_wait_cycles":59520,"thread_start_us":97,"threads_started":1}
I20260929 18:40:55.870052 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.850010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:49819)
0929 18:40:55.851061 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.851075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.851727 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.852270 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.852279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.853667 (+ 1388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.854446 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.854472 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.855686 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.855690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.857254 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.857265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.858052 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.858056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.858416 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.859485 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.859498 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.862520 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.864925 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.864935 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.864960 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.865482 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.865840 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.865845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.866059 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.866594 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.866602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.867062 (+ 460us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.868095 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.868627 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":258,"spinlock_wait_cycles":52608,"thread_start_us":146,"threads_started":1}
I20260929 18:40:55.871125 18947 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:55.871162 18946 heartbeater.cc:347] Connected to a master server at 127.11.118.252:43355
I20260929 18:40:55.871372 18947 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.871371 18946 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.872030 18946 heartbeater.cc:511] Master 127.11.118.252:43355 requested a full tablet report, sending...
I20260929 18:40:55.872030 18947 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
I20260929 18:40:55.873178 17844 ts_manager.cc:194] Re-registered known tserver with Master: 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
I20260929 18:40:55.873299 18624 ts_manager.cc:194] Registered new tserver with Master: 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
I20260929 18:40:55.873898 17844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:49819
I20260929 18:40:55.875346 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.869232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:41012)
0929 18:40:55.869526 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.869544 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.869633 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.870346 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.870352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.870374 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:55.870505 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.870515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.872069 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.872073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.873508 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.873520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.874943 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.874947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.875045 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.875067 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.875138 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.875221 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260929 18:40:55.875358 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.869801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.11.118.252:43355)
0929 18:40:55.869936 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:55.869943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:55.869958 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:55.870008 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:55.870014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:55.870082 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:55.870200 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:55.871109 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.871843 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.873832 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.874566 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.874675 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.874813 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:55.875174 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 18:40:55.875250 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:40:55.876255 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:55.850010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.11.118.194:42633)
0929 18:40:55.850647 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:55.850705 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:55.851727 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:55.852270 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:55.852279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:55.854748 (+ 2469us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:55.854969 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.854981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.856390 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.856396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.857557 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:55.857571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:55.858476 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:55.858482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:55.858603 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:55.859333 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:55.859360 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:55.862319 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:55.865095 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.865106 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.865561 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.865928 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:55.865934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:55.865939 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:55.866086 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:55.866706 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:55.866710 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:55.867066 (+ 356us) client_negotiation.cc:769] Sending connection context
0929 18:40:55.875803 (+ 8737us) client_negotiation.cc:240] Negotiation successful
0929 18:40:55.876052 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":67,"spinlock_wait_cycles":16896,"thread_start_us":177,"threads_started":1}
I20260929 18:40:55.876221 18634 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 18:40:55.876669 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f51d3eac064c56a2d30ff52c498988. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:55.876816 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: LEARNER
I20260929 18:40:55.876855 18948 heartbeater.cc:347] Connected to a master server at 127.11.118.253:33103
I20260929 18:40:55.876935 18948 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:55.877135 18948 heartbeater.cc:511] Master 127.11.118.253:33103 requested a full tablet report, sending...
I20260929 18:40:55.877873 18445 ts_manager.cc:194] Registered new tserver with Master: 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
W20260929 18:40:55.981459 18953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:40:55.981812 18953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:40:55.981910 18953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:40:55.986385 18953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:40:55.986481 18953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:40:55.986584 18953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:40:55.992848 18953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:46495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=42597
--tserver_master_addrs=127.11.118.254:39579,127.11.118.253:33103,127.11.118.252:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:40:55.994023 18953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:40:55.995162 18953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:40:56.005357 18959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:56.005367 18961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:40:56.005367 18958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:40:56.005689 18953 server_base.cc:1034] running on GCE node
I20260929 18:40:56.006213 18953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:40:56.006981 18953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:40:56.008196 18953 hybrid_clock.cc:648] HybridClock initialized: now 1790707256008178 us; error 33 us; skew 500 ppm
Sep 29 18:40:56 dist-test-slave-cgv7 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707256, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:40:56.011636 18953 init.cc:377] Logged in from keytab as oryx/127.11.118.195@KRBTEST.COM (short username oryx)
I20260929 18:40:56.012862 18953 webserver.cc:460] Webserver started at http://127.11.118.195:42597/ using document root <none> and password file <none>
I20260929 18:40:56.013484 18953 fs_manager.cc:359] Metadata directory not provided
I20260929 18:40:56.013604 18953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:40:56.017825 18953 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:40:56.020100 18968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:56.021461 18953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:40:56.021618 18953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea"
format_stamp: "Formatted at 2026-09-29 18:40:54 on dist-test-slave-cgv7"
server_key: "23eb9d8dfdcdbe684d20ebd0292bab01"
server_key_iv: "2b72618efb5602fa83b857a3d58ac8c5"
server_key_version: "encryptionkey@0"
I20260929 18:40:56.022198 18953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:40:56.039834 18953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:40:56.043341 18953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:40:56.043566 18953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:40:56.044178 18953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:40:56.045058 18953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:40:56.045132 18953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:56.045203 18953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:40:56.045259 18953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:40:56.055303 18953 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:46495
I20260929 18:40:56.055325 19081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:46495 every 8 connection(s)
I20260929 18:40:56.056316 18953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:40:56.057758 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 18953
Sep 29 18:40:56 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707256, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
Sep 29 18:40:56 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707256, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.252@KRBTEST.COM
Sep 29 18:40:56 dist-test-slave-cgv7 krb5kdc[17799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707256, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.253@KRBTEST.COM
I20260929 18:40:56.073310 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.060522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.11.118.254:39579)
0929 18:40:56.061297 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.061304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.061800 (+ 496us) server_negotiation.cc:407] Connection header received
0929 18:40:56.061839 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.061841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.061881 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.061957 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.063282 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.064077 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065422 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.065767 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065851 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.067970 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.067973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.067977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.067998 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.070907 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.071379 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.071382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.071384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.071461 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.071762 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.071766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.071767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.071990 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.072118 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.072915 (+ 797us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.073047 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":89,"threads_started":1}
I20260929 18:40:56.074046 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.060149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.11.118.253:33103)
0929 18:40:56.060560 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.060565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.061779 (+ 1214us) server_negotiation.cc:407] Connection header received
0929 18:40:56.061823 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.061826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.061870 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.061951 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.063281 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.064115 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065358 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.065707 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065825 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.067959 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.067962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.067965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.067990 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.070413 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.070869 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.070874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.070876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.070961 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.071289 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.071296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.071300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.071526 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.071646 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.073729 (+ 2083us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.073868 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":87,"threads_started":1}
I20260929 18:40:56.076439 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.058381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:37323 (local address 127.11.118.252:43355)
0929 18:40:56.058551 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.058556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.060302 (+ 1746us) server_negotiation.cc:407] Connection header received
0929 18:40:56.061161 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.061165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.061218 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.061267 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.062900 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.063660 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065297 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.065645 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065716 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.071567 (+ 5851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.071571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.071574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.071599 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.073731 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.074371 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.074374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.074376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.074456 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.074963 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.074969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.074974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.075261 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.075371 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.076140 (+ 769us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.076264 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260929 18:40:56.078109 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.058882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.11.118.195:37323)
0929 18:40:56.060221 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.060233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.060635 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.061415 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.061420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.061849 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:56.062779 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.062785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.063783 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.063786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.065173 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.065180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.065928 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.066112 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.067361 (+ 1249us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:56.067387 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:56.071383 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:56.073867 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.073873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.073890 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.074239 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.074594 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.074598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.074600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.074830 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.075400 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:56.075406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:56.075788 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.076633 (+ 845us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.077010 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":1351,"spinlock_wait_cycles":1280,"thread_start_us":92,"threads_started":1}
I20260929 18:40:56.078200 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.058486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:60499 (local address 127.11.118.254:39579)
0929 18:40:56.058725 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.058729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.060217 (+ 1488us) server_negotiation.cc:407] Connection header received
0929 18:40:56.061055 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.061058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.061114 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.061172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.064093 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.064836 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.065942 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.066238 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.066977 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.072074 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.072081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.072087 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.072115 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.075984 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.076450 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.076457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.076459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.076538 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.077042 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.077045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.077047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.077229 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.077304 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.077944 (+ 640us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.078050 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260929 18:40:56.079101 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.059889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:52199 (local address 127.11.118.253:33103)
0929 18:40:56.060022 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.060028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.060049 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:40:56.060861 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.060864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.060920 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.061005 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.062751 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.063582 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.067879 (+ 4297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.068229 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.068440 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.073116 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.073123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.073128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.073158 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.075835 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.077872 (+ 2037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.077875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.077877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.077949 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.078228 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.078231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.078233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.078451 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.078536 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.078856 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.078964 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:40:56.079365 19082 heartbeater.cc:347] Connected to a master server at 127.11.118.252:43355
I20260929 18:40:56.079569 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.058921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:60499)
0929 18:40:56.060128 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.060142 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.060929 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.061380 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.061386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.062452 (+ 1066us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:56.062581 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.062599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.064984 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.064987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.065801 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.065812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.067128 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.067131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.067232 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.067723 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:56.067737 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:56.071904 (+ 4167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:56.076095 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.076098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.076100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.076325 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.076839 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.076842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.076844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.076930 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.077370 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:56.077373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:56.077454 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.078685 (+ 1231us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.078839 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"thread_start_us":144,"threads_started":1}
I20260929 18:40:56.079625 19082 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:56.079576 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.058882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.11.118.195:52199)
0929 18:40:56.059406 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.059462 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.060630 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.061274 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.061283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.061972 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:40:56.062581 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.062599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.063761 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.063764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.067179 (+ 3415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.067185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.068895 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.068901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.069010 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.069952 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:40:56.069966 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:40:56.072965 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:40:56.077563 (+ 4598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.077566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.077568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.077778 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.078051 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:40:56.078054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:40:56.078056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:40:56.078134 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:40:56.078592 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:40:56.078595 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:40:56.078675 (+ 80us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.079270 (+ 595us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.079409 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":2394,"spinlock_wait_cycles":105216,"thread_start_us":144,"threads_started":1}
I20260929 18:40:56.080401 19082 heartbeater.cc:511] Master 127.11.118.252:43355 requested a full tablet report, sending...
I20260929 18:40:56.080487 19084 heartbeater.cc:347] Connected to a master server at 127.11.118.253:33103
I20260929 18:40:56.080555 19084 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:56.080690 19084 heartbeater.cc:511] Master 127.11.118.253:33103 requested a full tablet report, sending...
I20260929 18:40:56.077579 17844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 18:40:56.081234 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.060560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.11.118.252:43355)
0929 18:40:56.061096 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.061101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.064442 (+ 3341us) server_negotiation.cc:407] Connection header received
0929 18:40:56.066531 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.066534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.066583 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.066693 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.069224 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.070047 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.072106 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.072478 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.072554 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.075623 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.075626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.075629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.075655 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.077541 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.079886 (+ 2345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.079890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.079891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.079972 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.080284 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.080290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.080291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.080484 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.080590 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.080935 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.081080 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":104,"threads_started":1}
I20260929 18:40:56.081494 18445 ts_manager.cc:194] Registered new tserver with Master: 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:40:56.081548 18624 ts_manager.cc:194] Registered new tserver with Master: 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:40:56.082602 19083 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39579
I20260929 18:40:56.082683 19083 heartbeater.cc:464] Registering TS with master...
I20260929 18:40:56.082858 19083 heartbeater.cc:511] Master 127.11.118.254:39579 requested a full tablet report, sending...
W20260929 18:40:56.083401 17844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:40:56.083545 17841 ts_manager.cc:194] Re-registered known tserver with Master: 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:40:56.084273 17841 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:60499
I20260929 18:40:56.086347 18593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: NON_VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: true } } }
I20260929 18:40:56.086481 18593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:56.089466 19101 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: attempting to promote NON_VOTER fa3d239fdefa4ba89ec98265eedf0a0f to VOTER
I20260929 18:40:56.089777 19101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:56.091965 18454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 18:40:56.091976 18634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 LEARNER]: Refusing update from remote peer 71f51d3eac064c56a2d30ff52c498988: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 18:40:56.093713 18593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 18:40:56.094434 18593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 18:40:56.095456 19100 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.096362 18634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.096822 18454 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.097121 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.097244 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:56.097375 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.097451 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:56.097613 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.097689 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988 [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:56.098939 19109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.099138 19109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:40:56.101204 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71f51d3eac064c56a2d30ff52c498988" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:56.101349 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:40:56.106576 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.100638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.11.118.194:36905)
0929 18:40:56.101080 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.101093 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.101116 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:40:56.101196 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.101202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.101509 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.101672 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:56.102105 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.103342 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.105111 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.105951 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106080 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.106239 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.106319 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.106422 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":101,"threads_started":1}
I20260929 18:40:56.106570 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.100097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:42325 (local address 127.0.0.1:34660)
0929 18:40:56.100393 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.100408 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.100488 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.102185 (+ 1697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.102189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.102205 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:56.102344 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.102351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.103614 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.103617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.104809 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.104820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.106193 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.106318 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.106341 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.106390 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.106448 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":32,"thread_start_us":97,"threads_started":1}
I20260929 18:40:56.106568 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.100278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36905 (local address 127.0.0.1:59588)
0929 18:40:56.100775 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.100788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.100873 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.101845 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.101848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.101866 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:56.101972 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.101978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.103510 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.103513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.104992 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.105000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.106080 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.106168 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.106195 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.106306 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.106396 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":23,"thread_start_us":96,"threads_started":1}
I20260929 18:40:56.107085 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.100425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.11.118.193:42325)
0929 18:40:56.101257 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.101271 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.101291 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:40:56.101355 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.101360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.102041 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.102199 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:56.102449 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.103503 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.105045 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.106065 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106660 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.106786 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.106834 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.106931 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":90,"threads_started":1}
I20260929 18:40:56.107252 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.099850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:46495 (local address 127.0.0.1:44082)
0929 18:40:56.100344 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.100362 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.100458 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.103007 (+ 2549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.103010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.103030 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:56.103124 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.103131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.104505 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.104509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.105503 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.105512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.106540 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.106984 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.107010 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.107088 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.107151 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415}
I20260929 18:40:56.107657 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.100000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.11.118.195:46495)
0929 18:40:56.102501 (+ 2501us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.102514 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.102535 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:40:56.102617 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.102621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.102801 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.103285 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:56.103444 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.104358 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.105627 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.106247 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.106374 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.106483 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.107426 (+ 943us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.107523 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2395,"thread_start_us":124,"threads_started":1}
I20260929 18:40:56.109542 18880 tablet_service.cc:1467] Processing CreateTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 (DEFAULT_TABLE table=test-table [id=0eea89f49a5843aaac8eae60553b79d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:56.109696 18743 tablet_service.cc:1467] Processing CreateTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 (DEFAULT_TABLE table=test-table [id=0eea89f49a5843aaac8eae60553b79d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:56.110170 19016 tablet_service.cc:1467] Processing CreateTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 (DEFAULT_TABLE table=test-table [id=0eea89f49a5843aaac8eae60553b79d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:40:56.110607 18880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c78eb93137b4dc4af377da87a5d9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:56.110713 18743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c78eb93137b4dc4af377da87a5d9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:56.111064 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c78eb93137b4dc4af377da87a5d9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:40:56.118966 19116 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Bootstrap starting.
I20260929 18:40:56.120929 19116 tablet_bootstrap.cc:654] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:56.121670 19116 log.cc:824] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:56.122406 19115 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Bootstrap starting.
I20260929 18:40:56.122625 19117 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Bootstrap starting.
I20260929 18:40:56.123769 19116 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: No bootstrap required, opened a new log
I20260929 18:40:56.123988 19116 ts_tablet_manager.cc:1397] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 18:40:56.124701 19115 tablet_bootstrap.cc:654] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:56.124701 19117 tablet_bootstrap.cc:654] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Neither blocks nor log segments found. Creating new log.
I20260929 18:40:56.125556 19115 log.cc:824] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:56.125784 19117 log.cc:824] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Log is configured to *not* fsync() on all Append() calls
I20260929 18:40:56.127760 19115 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: No bootstrap required, opened a new log
I20260929 18:40:56.127879 19117 tablet_bootstrap.cc:492] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: No bootstrap required, opened a new log
I20260929 18:40:56.127954 19115 ts_tablet_manager.cc:1397] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:40:56.128065 19117 ts_tablet_manager.cc:1397] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:40:56.128556 19116 raft_consensus.cc:348] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.128809 19116 raft_consensus.cc:374] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:56.128863 19116 raft_consensus.cc:729] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54790de3458470ba4da284f400e5bc8, State: Initialized, Role: FOLLOWER
I20260929 18:40:56.129307 19116 consensus_queue.cc:260] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.130139 19116 ts_tablet_manager.cc:1428] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:40:56.131459 18810 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
I20260929 18:40:56.133774 19115 raft_consensus.cc:348] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.134063 19115 raft_consensus.cc:374] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:56.134126 19115 raft_consensus.cc:729] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdf123505e4677a00bdf8b2e1e2201, State: Initialized, Role: FOLLOWER
I20260929 18:40:56.134653 19115 consensus_queue.cc:260] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.134691 19117 raft_consensus.cc:348] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.134943 19117 raft_consensus.cc:374] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:40:56.135007 19117 raft_consensus.cc:729] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2ba5bf0ab84dd190fe8ffe19db51ea, State: Initialized, Role: FOLLOWER
I20260929 18:40:56.135401 18947 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
I20260929 18:40:56.135464 19117 consensus_queue.cc:260] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.135740 19115 ts_tablet_manager.cc:1428] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260929 18:40:56.136369 19117 ts_tablet_manager.cc:1428] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260929 18:40:56.137610 19083 heartbeater.cc:503] Master 127.11.118.254:39579 was elected leader, sending a full tablet report...
I20260929 18:40:56.248116 19123 raft_consensus.cc:482] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:56.248404 19123 raft_consensus.cc:504] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.249583 19123 leader_election.cc:288] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325), 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
I20260929 18:40:56.253515 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.250113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:42461 (local address 127.11.118.193:42325)
0929 18:40:56.250298 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.250303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.250315 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:40:56.250362 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.250364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.250401 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.250462 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:56.250725 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.251236 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.252509 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.252908 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.253033 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.253086 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.253305 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.253377 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:40:56.253516 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.249940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:50943 (local address 127.11.118.194:36905)
0929 18:40:56.250057 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.250062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.250077 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:56.250575 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.250578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.250616 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.250706 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:56.250976 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.251476 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.252522 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.252938 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.253065 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.253112 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.253305 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.253384 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:40:56.253516 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.249854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:42325 (local address 127.11.118.195:42461)
0929 18:40:56.250058 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.250073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.250158 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.250491 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.250493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.250508 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:56.250607 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.250612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.251361 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.251367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.252386 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.252395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.253033 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.253036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.253152 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.253198 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.253284 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.253377 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":197}
I20260929 18:40:56.253531 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.249850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36905 (local address 127.11.118.195:50943)
0929 18:40:56.249970 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:56.249986 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:56.250089 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:56.250718 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:56.250729 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:56.250745 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:56.250866 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.250871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.251591 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.251596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.252398 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:56.252407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.253058 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.253062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:56.253163 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.253203 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 18:40:56.253289 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:40:56.253388 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":47}
I20260929 18:40:56.254205 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c78eb93137b4dc4af377da87a5d9f52" candidate_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54790de3458470ba4da284f400e5bc8" is_pre_election: true
I20260929 18:40:56.254339 18900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c78eb93137b4dc4af377da87a5d9f52" candidate_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdf123505e4677a00bdf8b2e1e2201" is_pre_election: true
I20260929 18:40:56.254576 18900 raft_consensus.cc:2454] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2ba5bf0ab84dd190fe8ffe19db51ea in term 0.
I20260929 18:40:56.254576 18763 raft_consensus.cc:2454] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2ba5bf0ab84dd190fe8ffe19db51ea in term 0.
I20260929 18:40:56.255118 18970 leader_election.cc:302] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a2ba5bf0ab84dd190fe8ffe19db51ea, 71bdf123505e4677a00bdf8b2e1e2201, f54790de3458470ba4da284f400e5bc8; no voters:
I20260929 18:40:56.255393 19123 raft_consensus.cc:2781] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:40:56.255471 19123 raft_consensus.cc:482] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:56.255513 19123 raft_consensus.cc:3037] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:56.256529 19123 raft_consensus.cc:504] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.256891 19123 leader_election.cc:288] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [CANDIDATE]: Term 1 election: Requested vote from peers f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325), 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905)
I20260929 18:40:56.257182 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c78eb93137b4dc4af377da87a5d9f52" candidate_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54790de3458470ba4da284f400e5bc8"
I20260929 18:40:56.257339 18900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c78eb93137b4dc4af377da87a5d9f52" candidate_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdf123505e4677a00bdf8b2e1e2201"
I20260929 18:40:56.257361 18763 raft_consensus.cc:3037] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:56.257480 18900 raft_consensus.cc:3037] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:40:56.258554 18763 raft_consensus.cc:2454] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2ba5bf0ab84dd190fe8ffe19db51ea in term 1.
I20260929 18:40:56.258538 18900 raft_consensus.cc:2454] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2ba5bf0ab84dd190fe8ffe19db51ea in term 1.
I20260929 18:40:56.258839 18971 leader_election.cc:302] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2ba5bf0ab84dd190fe8ffe19db51ea, 71bdf123505e4677a00bdf8b2e1e2201; no voters:
I20260929 18:40:56.259037 19123 raft_consensus.cc:2781] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:40:56.259282 19123 raft_consensus.cc:686] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 LEADER]: Becoming Leader. State: Replica: 2a2ba5bf0ab84dd190fe8ffe19db51ea, State: Running, Role: LEADER
I20260929 18:40:56.259629 19123 consensus_queue.cc:237] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } }
I20260929 18:40:56.263089 17844 catalog_manager.cc:5495] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195). New cstate: current_term: 1 leader_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20260929 18:40:56.299754 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.296315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.11.118.195:46495)
0929 18:40:56.296440 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.296443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.296456 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:40:56.296530 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.296532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.296585 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.296683 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:40:56.296924 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.297428 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.298283 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.298502 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.298592 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.298864 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:40:56.299282 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:40:56.299372 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.299488 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.299579 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 18:40:56.305953 18763 raft_consensus.cc:1261] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Refusing update from remote peer 2a2ba5bf0ab84dd190fe8ffe19db51ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:56.305961 18900 raft_consensus.cc:1261] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Refusing update from remote peer 2a2ba5bf0ab84dd190fe8ffe19db51ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:40:56.306769 19124 consensus_queue.cc:1035] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:40:56.307046 19123 consensus_queue.cc:1035] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:40:56.307514 19085 tablet.cc:2369] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:40:56.316088 19128 mvcc.cc:205] Tried to move back new op lower bound from 7334736921821851648 to 7334736921642299392. Current Snapshot: MvccSnapshot[applied={T|T < 7334736921821851648}]
I20260929 18:40:56.328608 17844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:56.328848 17844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:40:56.333927 17844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71f51d3eac064c56a2d30ff52c498988: Sending DeleteTablet for 3 replicas of tablet 7c78eb93137b4dc4af377da87a5d9f52
I20260929 18:40:56.334849 18743 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:56 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34660
I20260929 18:40:56.334937 18880 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:56 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:59588
I20260929 18:40:56.334961 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c78eb93137b4dc4af377da87a5d9f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:40:56 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:44082
I20260929 18:40:56.335425 19138 tablet_replica.cc:330] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: stopping tablet replica
I20260929 18:40:56.337270 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 17812
I20260929 18:40:56.342360 19138 raft_consensus.cc:2229] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:56.342794 19138 raft_consensus.cc:2258] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:56.343238 19137 tablet_replica.cc:330] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: stopping tablet replica
I20260929 18:40:56.343618 19137 raft_consensus.cc:2229] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:40:56.343947 19137 raft_consensus.cc:2258] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:56.344343 19139 tablet_replica.cc:330] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: stopping tablet replica
I20260929 18:40:56.344626 19139 raft_consensus.cc:2229] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:40:56.344666 19138 ts_tablet_manager.cc:1905] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:56.345036 19139 raft_consensus.cc:2258] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:40:56.346890 19139 ts_tablet_manager.cc:1905] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:56.347818 19137 ts_tablet_manager.cc:1905] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:40:56.349643 19139 ts_tablet_manager.cc:1918] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:56.349753 19139 log.cc:1196] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/7c78eb93137b4dc4af377da87a5d9f52
I20260929 18:40:56.350098 19139 ts_tablet_manager.cc:1939] T 7c78eb93137b4dc4af377da87a5d9f52 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting consensus metadata
W20260929 18:40:56.351145 18971 connection.cc:441] server connection from 127.0.0.1:44082 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44082 (request call id 1) could send its response
I20260929 18:40:56.351755 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.337159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.11.118.253:33103)
0929 18:40:56.337284 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.337289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.337304 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:56.337360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.337362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.337407 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.337477 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.338966 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.339769 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.340979 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.341336 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.341420 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.343702 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.343709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.343716 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.343752 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.346178 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.349890 (+ 3712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.349895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.349896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.350003 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.350479 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.350483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.350485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.350747 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.350879 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.351407 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.351541 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:40:56.353529 19138 ts_tablet_manager.cc:1918] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:56.353642 19138 log.cc:1196] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/7c78eb93137b4dc4af377da87a5d9f52
I20260929 18:40:56.353801 19137 ts_tablet_manager.cc:1918] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:40:56.353960 19137 log.cc:1196] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/7c78eb93137b4dc4af377da87a5d9f52
I20260929 18:40:56.353984 19138 ts_tablet_manager.cc:1939] T 7c78eb93137b4dc4af377da87a5d9f52 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting consensus metadata
I20260929 18:40:56.354411 19137 ts_tablet_manager.cc:1939] T 7c78eb93137b4dc4af377da87a5d9f52 P f54790de3458470ba4da284f400e5bc8: Deleting consensus metadata
W20260929 18:40:56.355186 18834 connection.cc:441] server connection from 127.0.0.1:59588 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59588 (request call id 1) could send its response
W20260929 18:40:56.355549 18698 connection.cc:441] server connection from 127.0.0.1:34660 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34660 (request call id 1) could send its response
I20260929 18:40:56.362726 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.349792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.11.118.253:33103)
0929 18:40:56.349915 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.349919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.350915 (+ 996us) server_negotiation.cc:407] Connection header received
0929 18:40:56.351078 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.351082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.351135 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.351187 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.354225 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.355053 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.356432 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.356756 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.356859 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.358695 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.358699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.358701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.358727 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.360648 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.361128 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.361132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.361134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.361222 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.361556 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.361561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.361563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.361808 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.361919 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.362368 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.362514 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:40:56.365870 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.337035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.11.118.252:43355)
0929 18:40:56.337297 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.337302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.352741 (+ 15439us) server_negotiation.cc:407] Connection header received
0929 18:40:56.352804 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.352807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.352858 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.352930 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.356547 (+ 3617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.357333 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.358621 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.359020 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.359109 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.360984 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.360989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.360993 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.361245 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.363237 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.364474 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.364477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.364479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.364562 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.364860 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.364866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.364871 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.365053 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.365191 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.365431 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.365693 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260929 18:40:56.365870 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:56.354001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.11.118.252:43355)
0929 18:40:56.354138 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:56.354144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:56.357093 (+ 2949us) server_negotiation.cc:407] Connection header received
0929 18:40:56.357139 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:56.357142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:56.357189 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:56.357260 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:40:56.358235 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.359016 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.360208 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:56.360543 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:56.360612 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:56.362253 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:40:56.362257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:40:56.362260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:40:56.362310 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:40:56.363890 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.364551 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.364554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.364555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.364624 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:40:56.364901 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:40:56.364904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:40:56.364906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:40:56.365106 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:40:56.365222 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:56.365529 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 18:40:56.365755 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":51}
I20260929 18:40:57.318578 19148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.318006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:42419)
0929 18:40:57.318333 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.318434 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
W20260929 18:40:57.318990 18947 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:39579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.321177 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.320697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:46033)
0929 18:40:57.320970 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.321062 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
W20260929 18:40:57.321501 18810 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:39579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.321542 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.320998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:46087)
0929 18:40:57.321211 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.321461 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
W20260929 18:40:57.321789 19083 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:39579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.833662 19151 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71f51d3eac064c56a2d30ff52c498988)
I20260929 18:40:57.833842 19152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71f51d3eac064c56a2d30ff52c498988)
I20260929 18:40:57.833860 19151 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:57.834049 19152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:57.835197 19152 leader_election.cc:288] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252:43355)
I20260929 18:40:57.835371 19151 leader_election.cc:288] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253:33103)
I20260929 18:40:57.835743 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.835076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58190)
0929 18:40:57.835564 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.835636 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":61,"threads_started":1}
W20260929 18:40:57.835908 18615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.836962 19159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.835757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58192)
0929 18:40:57.836776 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.836856 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":62,"threads_started":1}
W20260929 18:40:57.837247 18430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.838160 19159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.837912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58204)
0929 18:40:57.838051 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.838098 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260929 18:40:57.838347 18430 leader_election.cc:334] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.838519 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.836539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58200)
0929 18:40:57.838397 (+ 1858us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.838448 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1800}
W20260929 18:40:57.838645 18615 leader_election.cc:334] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.841421 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.835076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33103 (local address 127.0.0.1:56440)
0929 18:40:57.836336 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.836352 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:57.836683 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:57.837146 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:57.837152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:57.837178 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:57.837328 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:57.837336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.838618 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.838622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:57.840068 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:57.840081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.841025 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.841028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:57.841118 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:57.841142 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 18:40:57.841210 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 18:40:57.841288 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1206,"mutex_wait_us":217,"spinlock_wait_cycles":10624,"thread_start_us":60,"threads_started":1}
I20260929 18:40:57.841423 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.835227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.11.118.253:33103)
0929 18:40:57.836837 (+ 1610us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:57.836844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:57.836862 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:40:57.836919 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:57.836922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:57.836981 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:57.837447 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:57.837596 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.838491 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.840229 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.840858 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.841035 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:57.841181 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:57.841222 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 18:40:57.841288 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":1523,"spinlock_wait_cycles":1792,"thread_start_us":54,"threads_started":1}
I20260929 18:40:57.841635 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.836510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:43355 (local address 127.0.0.1:41064)
0929 18:40:57.836859 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.836871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:40:57.836949 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:40:57.837456 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:40:57.837462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:40:57.837494 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:40:57.837866 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:57.837874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.839131 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.839138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:57.839937 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:40:57.839946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.840858 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.840864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:40:57.841118 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:57.841184 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 18:40:57.841300 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 18:40:57.841395 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":266,"spinlock_wait_cycles":12288,"thread_start_us":75,"threads_started":1}
I20260929 18:40:57.841887 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.836277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.11.118.252:43355)
0929 18:40:57.837130 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0929 18:40:57.837139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 18:40:57.837154 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:40:57.837197 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:40:57.837199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:40:57.837270 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:40:57.837349 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:40:57.837995 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.839027 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.840084 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:40:57.840710 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:40:57.840837 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:40:57.840895 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 18:40:57.841690 (+ 795us) server_negotiation.cc:299] Negotiation successful
0929 18:40:57.841768 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"spinlock_wait_cycles":3072,"thread_start_us":100,"threads_started":1}
I20260929 18:40:57.842079 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5194fde6067d4b9c968d19dd3e639178" is_pre_election: true
I20260929 18:40:57.842330 18454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3d239fdefa4ba89ec98265eedf0a0f in term 1.
I20260929 18:40:57.842624 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5194fde6067d4b9c968d19dd3e639178" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" is_pre_election: true
I20260929 18:40:57.842706 18617 leader_election.cc:302] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5194fde6067d4b9c968d19dd3e639178, fa3d239fdefa4ba89ec98265eedf0a0f; no voters: 71f51d3eac064c56a2d30ff52c498988
I20260929 18:40:57.842854 18634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5194fde6067d4b9c968d19dd3e639178 in term 1.
I20260929 18:40:57.843046 19151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 18:40:57.843106 19151 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71f51d3eac064c56a2d30ff52c498988)
I20260929 18:40:57.843151 19151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:40:57.843248 18429 leader_election.cc:302] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5194fde6067d4b9c968d19dd3e639178, fa3d239fdefa4ba89ec98265eedf0a0f; no voters: 71f51d3eac064c56a2d30ff52c498988
I20260929 18:40:57.843577 19152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 18:40:57.843626 19152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71f51d3eac064c56a2d30ff52c498988)
I20260929 18:40:57.843660 19152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:40:57.844480 19151 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:57.844962 19151 leader_election.cc:288] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 election: Requested vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253:33103)
I20260929 18:40:57.844944 19152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:57.845268 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5194fde6067d4b9c968d19dd3e639178"
I20260929 18:40:57.845407 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.845108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58210)
0929 18:40:57.845244 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.845303 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 18:40:57.845431 18454 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fa3d239fdefa4ba89ec98265eedf0a0f in current term 2: Already voted for candidate 5194fde6067d4b9c968d19dd3e639178 in this term.
I20260929 18:40:57.845656 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.845395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58212)
0929 18:40:57.845537 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.845593 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 18:40:57.845898 19152 leader_election.cc:288] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 election: Requested vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), fa3d239fdefa4ba89ec98265eedf0a0f (127.11.118.252:43355)
I20260929 18:40:57.846050 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.845839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58220)
0929 18:40:57.845939 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.845990 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 18:40:57.846179 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5194fde6067d4b9c968d19dd3e639178" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f"
I20260929 18:40:57.846361 18634 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5194fde6067d4b9c968d19dd3e639178 in current term 2: Already voted for candidate fa3d239fdefa4ba89ec98265eedf0a0f in this term.
I20260929 18:40:57.846495 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:57.846242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58228)
0929 18:40:57.846360 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:57.846408 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 18:40:57.846577 18615 leader_election.cc:334] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
W20260929 18:40:57.846632 18430 leader_election.cc:334] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:57.846644 18615 leader_election.cc:302] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3d239fdefa4ba89ec98265eedf0a0f; no voters: 5194fde6067d4b9c968d19dd3e639178, 71f51d3eac064c56a2d30ff52c498988
I20260929 18:40:57.846797 19151 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 18:40:57.846867 18429 leader_election.cc:302] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5194fde6067d4b9c968d19dd3e639178; no voters: 71f51d3eac064c56a2d30ff52c498988, fa3d239fdefa4ba89ec98265eedf0a0f
I20260929 18:40:57.847003 19152 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 18:40:58.320225 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:58.319668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:41071)
0929 18:40:58.320020 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:58.320098 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":104,"threads_started":1}
I20260929 18:40:58.322638 19162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:58.322104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:60739)
0929 18:40:58.322489 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:58.322549 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":134,"threads_started":1}
I20260929 18:40:58.322886 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:58.322415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:44585)
0929 18:40:58.322719 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:58.322782 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":80,"threads_started":1}
I20260929 18:40:59.321880 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.321381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:46039)
0929 18:40:59.321693 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.321768 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":70,"threads_started":1}
I20260929 18:40:59.323824 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.323365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:48163)
0929 18:40:59.323644 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.323718 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":63,"threads_started":1}
I20260929 18:40:59.323920 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.323536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:53253)
0929 18:40:59.323744 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.323818 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260929 18:40:59.957715 19167 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:40:59.957875 19167 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:59.958402 19167 leader_election.cc:288] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253:33103)
I20260929 18:40:59.958904 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5194fde6067d4b9c968d19dd3e639178" is_pre_election: true
I20260929 18:40:59.959040 18454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3d239fdefa4ba89ec98265eedf0a0f in term 2.
I20260929 18:40:59.959081 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.958630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58562)
0929 18:40:59.958886 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.958954 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":117,"threads_started":1}
I20260929 18:40:59.959437 18617 leader_election.cc:302] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5194fde6067d4b9c968d19dd3e639178, fa3d239fdefa4ba89ec98265eedf0a0f; no voters:
I20260929 18:40:59.959709 19167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260929 18:40:59.959820 19167 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:40:59.959877 19167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 2 FOLLOWER]: Advancing to term 3
I20260929 18:40:59.959959 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.959750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58576)
0929 18:40:59.959841 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.959895 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 18:40:59.960108 18615 leader_election.cc:334] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:59.961014 19167 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:59.961380 19167 leader_election.cc:288] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 election: Requested vote from peers 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579), 5194fde6067d4b9c968d19dd3e639178 (127.11.118.253:33103)
I20260929 18:40:59.961660 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.961470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58582)
0929 18:40:59.961551 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.961613 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 18:40:59.961820 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5194fde6067d4b9c968d19dd3e639178"
I20260929 18:40:59.961936 18454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 2 FOLLOWER]: Advancing to term 3
I20260929 18:40:59.962962 18454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3d239fdefa4ba89ec98265eedf0a0f in term 3.
I20260929 18:40:59.963295 18617 leader_election.cc:302] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5194fde6067d4b9c968d19dd3e639178, fa3d239fdefa4ba89ec98265eedf0a0f; no voters:
I20260929 18:40:59.963470 19167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 3 FOLLOWER]: Leader election won for term 3
I20260929 18:40:59.963722 19167 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [term 3 LEADER]: Becoming Leader. State: Replica: fa3d239fdefa4ba89ec98265eedf0a0f, State: Running, Role: LEADER
I20260929 18:40:59.963960 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:40:59.963733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58594)
0929 18:40:59.963852 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 18:40:59.963904 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 18:40:59.964099 18615 leader_election.cc:334] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
I20260929 18:40:59.964069 19167 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } }
I20260929 18:40:59.965173 19170 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa3d239fdefa4ba89ec98265eedf0a0f. Latest consensus state: current_term: 3 leader_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:40:59.965296 19170 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: LEADER
I20260929 18:40:59.965566 19172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:40:59.967715 19172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:40:59.968092 19172 catalog_manager.cc:1259] Loaded cluster ID: 7baea67fa1604e70b6ce11601db023ab
I20260929 18:40:59.968168 19172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:40:59.968708 19172 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:40:59.969087 19172 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Loaded TSK: 0
I20260929 18:40:59.969745 19172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:40:59.998721 18624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:41:00.002002 18624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:00.005542 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.005192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58606)
0929 18:41:00.005378 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.005451 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 18:41:00.005620 18454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [term 3 FOLLOWER]: Refusing update from remote peer fa3d239fdefa4ba89ec98265eedf0a0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260929 18:41:00.006366 19167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 18:41:00.006534 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.006265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.0.0.1:58618)
0929 18:41:00.006405 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.006470 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 18:41:00.006708 18615 consensus_peers.cc:487] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f -> Peer 71f51d3eac064c56a2d30ff52c498988 (127.11.118.254:39579): Couldn't send request to peer 71f51d3eac064c56a2d30ff52c498988. Status: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 18:41:00.008152 19174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa3d239fdefa4ba89ec98265eedf0a0f. Latest consensus state: current_term: 3 leader_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:41:00.008322 19174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:41:00.010311 19174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:41:00.010460 19174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194fde6067d4b9c968d19dd3e639178 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:41:00.010664 19170 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "71f51d3eac064c56a2d30ff52c498988" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39579 } } peers { permanent_uuid: "5194fde6067d4b9c968d19dd3e639178" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33103 } attrs { promote: false } } peers { permanent_uuid: "fa3d239fdefa4ba89ec98265eedf0a0f" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 43355 } attrs { promote: false } } }
I20260929 18:41:00.010798 19170 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:00.020488 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.016161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:36905 (local address 127.0.0.1:59594)
0929 18:41:00.016344 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.016361 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.016455 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.016899 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:00.016902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:00.016919 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:00.017046 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.017053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018151 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.018154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.019113 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.019122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.019994 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.019998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.020165 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.020199 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 18:41:00.020274 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 18:41:00.020344 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":39}
I20260929 18:41:00.020527 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.016277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.11.118.194:36905)
0929 18:41:00.016636 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.016642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.016657 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:00.016715 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.016718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.016768 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.017179 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:00.017305 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018052 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.019253 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.019876 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.019999 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.020058 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.020309 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.020393 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":76,"threads_started":1}
I20260929 18:41:00.021270 18880 tablet_service.cc:1467] Processing CreateTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 (DEFAULT_TABLE table=test-table [id=8fef290cf003449dbd69b08ad344da99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:00.021462 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.016574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:42325 (local address 127.0.0.1:34662)
0929 18:41:00.016819 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.016832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.017002 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.017719 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:00.017721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:00.017737 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:00.017819 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.017824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018860 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.018864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.020048 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.020057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.021042 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.021046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.021127 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.021150 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:41:00.021240 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 18:41:00.021306 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":26,"thread_start_us":124,"threads_started":1}
I20260929 18:41:00.021598 18880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de3f917b66a4dc7ab41a9e84375a701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:00.021780 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.016905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:46495 (local address 127.0.0.1:44090)
0929 18:41:00.017357 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.017369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.017446 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.017902 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:00.017906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:00.017924 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:00.018004 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.018009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018944 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.018949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.020151 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.020159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.021252 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.021255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.021327 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.021346 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 18:41:00.021411 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 18:41:00.021462 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":55,"thread_start_us":81,"threads_started":1}
I20260929 18:41:00.021770 19182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.017221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.11.118.193:42325)
0929 18:41:00.017484 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.017489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.017503 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:00.017552 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.017555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.017607 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.017697 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:00.017940 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018748 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.020146 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.020792 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.020926 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.020994 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.021527 (+ 533us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.021621 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260929 18:41:00.022233 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.017343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.11.118.195:46495)
0929 18:41:00.017563 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.017567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.017579 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:41:00.017626 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.017632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.017687 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.017762 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:00.018124 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.018864 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.020294 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.021133 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.021248 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.021314 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.021492 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.021741 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
I20260929 18:41:00.022251 18743 tablet_service.cc:1467] Processing CreateTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 (DEFAULT_TABLE table=test-table [id=8fef290cf003449dbd69b08ad344da99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:00.022575 18743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de3f917b66a4dc7ab41a9e84375a701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:00.022742 19016 tablet_service.cc:1467] Processing CreateTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 (DEFAULT_TABLE table=test-table [id=8fef290cf003449dbd69b08ad344da99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:00.022989 19016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de3f917b66a4dc7ab41a9e84375a701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:00.024058 19184 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Bootstrap starting.
I20260929 18:41:00.025012 19184 tablet_bootstrap.cc:654] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:00.025101 19185 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: Bootstrap starting.
I20260929 18:41:00.025411 19186 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Bootstrap starting.
I20260929 18:41:00.025935 19185 tablet_bootstrap.cc:654] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:00.026445 19186 tablet_bootstrap.cc:654] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:00.027151 19184 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: No bootstrap required, opened a new log
I20260929 18:41:00.027334 19184 ts_tablet_manager.cc:1397] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:00.027508 19185 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: No bootstrap required, opened a new log
I20260929 18:41:00.027593 19185 ts_tablet_manager.cc:1397] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:00.027858 19184 raft_consensus.cc:348] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.027980 19184 raft_consensus.cc:374] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:00.028015 19184 raft_consensus.cc:729] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdf123505e4677a00bdf8b2e1e2201, State: Initialized, Role: FOLLOWER
I20260929 18:41:00.028028 19185 raft_consensus.cc:348] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.028162 19184 consensus_queue.cc:260] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.028517 19184 ts_tablet_manager.cc:1428] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:00.028599 19186 tablet_bootstrap.cc:492] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: No bootstrap required, opened a new log
I20260929 18:41:00.028715 19186 ts_tablet_manager.cc:1397] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:00.029402 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.028810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:43525)
0929 18:41:00.029237 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.029302 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
I20260929 18:41:00.029466 19186 raft_consensus.cc:348] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.029634 19186 raft_consensus.cc:374] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:00.029745 19185 raft_consensus.cc:374] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:00.029838 19185 raft_consensus.cc:729] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54790de3458470ba4da284f400e5bc8, State: Initialized, Role: FOLLOWER
I20260929 18:41:00.029922 19186 raft_consensus.cc:729] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2ba5bf0ab84dd190fe8ffe19db51ea, State: Initialized, Role: FOLLOWER
I20260929 18:41:00.030100 19185 consensus_queue.cc:260] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.030402 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.029981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:60089)
0929 18:41:00.030249 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.030332 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
I20260929 18:41:00.030416 19186 consensus_queue.cc:260] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.030855 19185 ts_tablet_manager.cc:1428] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 18:41:00.031181 18946 heartbeater.cc:503] Master 127.11.118.252:43355 was elected leader, sending a full tablet report...
I20260929 18:41:00.031762 19186 ts_tablet_manager.cc:1428] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 18:41:00.032627 19082 heartbeater.cc:503] Master 127.11.118.252:43355 was elected leader, sending a full tablet report...
I20260929 18:41:00.032779 19190 raft_consensus.cc:482] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:00.032873 19190 raft_consensus.cc:504] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.033448 18809 heartbeater.cc:503] Master 127.11.118.252:43355 was elected leader, sending a full tablet report...
I20260929 18:41:00.034118 19190 leader_election.cc:288] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325), 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:41:00.034613 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.031444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:50937)
0929 18:41:00.034456 (+ 3012us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.034517 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2929,"thread_start_us":84,"threads_started":1}
I20260929 18:41:00.035172 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.034957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:50839)
0929 18:41:00.035072 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.035115 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:41:00.037264 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.032303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:56045)
0929 18:41:00.037090 (+ 4787us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.037149 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4695,"thread_start_us":84,"threads_started":1}
I20260929 18:41:00.037972 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.037730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:51603)
0929 18:41:00.037853 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.037904 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 18:41:00.039423 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.034562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:42325 (local address 127.11.118.194:38421)
0929 18:41:00.034927 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.034942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.035029 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.035716 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:00.035720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:00.035736 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:00.035856 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.035862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.036812 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.036815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.038211 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.038220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.039113 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.039116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.039187 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.039207 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 18:41:00.039270 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 18:41:00.039336 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260929 18:41:00.039469 19182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.035381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38421 (local address 127.11.118.193:42325)
0929 18:41:00.035498 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.035504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.035516 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:41:00.035561 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.035565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.035604 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.035706 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:00.035976 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.036688 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.038384 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.038976 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.039094 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.039142 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.039301 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.039365 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:41:00.039960 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de3f917b66a4dc7ab41a9e84375a701" candidate_uuid: "71bdf123505e4677a00bdf8b2e1e2201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54790de3458470ba4da284f400e5bc8" is_pre_election: true
I20260929 18:41:00.040100 18763 raft_consensus.cc:2454] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdf123505e4677a00bdf8b2e1e2201 in term 0.
I20260929 18:41:00.040442 18834 leader_election.cc:302] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdf123505e4677a00bdf8b2e1e2201, f54790de3458470ba4da284f400e5bc8; no voters:
I20260929 18:41:00.040737 19190 raft_consensus.cc:2781] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:00.040798 19190 raft_consensus.cc:482] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:00.040830 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.035449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:56199 (local address 127.11.118.195:46495)
0929 18:41:00.035544 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.035548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.036880 (+ 1332us) server_negotiation.cc:407] Connection header received
0929 18:41:00.036958 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.036961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.037002 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.037080 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:00.037406 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.038097 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.039750 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.040319 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.040407 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.040441 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.040685 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.040744 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 18:41:00.040853 19190 raft_consensus.cc:3037] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:00.041021 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.034793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:46495 (local address 127.11.118.194:56199)
0929 18:41:00.036790 (+ 1997us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.036804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.036894 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.037120 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:00.037125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:00.037144 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:00.037264 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.037273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.038238 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.038241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.039407 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:00.039416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.040421 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.040424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:00.040523 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.040553 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 18:41:00.040664 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 18:41:00.040740 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1917}
I20260929 18:41:00.041536 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de3f917b66a4dc7ab41a9e84375a701" candidate_uuid: "71bdf123505e4677a00bdf8b2e1e2201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" is_pre_election: true
I20260929 18:41:00.041745 19036 raft_consensus.cc:2454] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdf123505e4677a00bdf8b2e1e2201 in term 0.
I20260929 18:41:00.042348 19190 raft_consensus.cc:504] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.042742 19190 leader_election.cc:288] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [CANDIDATE]: Term 1 election: Requested vote from peers f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325), 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495)
I20260929 18:41:00.043130 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de3f917b66a4dc7ab41a9e84375a701" candidate_uuid: "71bdf123505e4677a00bdf8b2e1e2201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea"
I20260929 18:41:00.043146 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de3f917b66a4dc7ab41a9e84375a701" candidate_uuid: "71bdf123505e4677a00bdf8b2e1e2201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54790de3458470ba4da284f400e5bc8"
I20260929 18:41:00.043241 19036 raft_consensus.cc:3037] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:00.043279 18763 raft_consensus.cc:3037] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:00.044230 18763 raft_consensus.cc:2454] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdf123505e4677a00bdf8b2e1e2201 in term 1.
I20260929 18:41:00.044242 19036 raft_consensus.cc:2454] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdf123505e4677a00bdf8b2e1e2201 in term 1.
I20260929 18:41:00.044550 18834 leader_election.cc:302] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2ba5bf0ab84dd190fe8ffe19db51ea, 71bdf123505e4677a00bdf8b2e1e2201; no voters:
I20260929 18:41:00.044785 19190 raft_consensus.cc:2781] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:00.045037 19190 raft_consensus.cc:686] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 LEADER]: Becoming Leader. State: Replica: 71bdf123505e4677a00bdf8b2e1e2201, State: Running, Role: LEADER
I20260929 18:41:00.045310 19190 consensus_queue.cc:237] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } }
I20260929 18:41:00.047240 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.046877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:54209)
0929 18:41:00.047049 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.047160 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 18:41:00.047771 18624 catalog_manager.cc:5495] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "71bdf123505e4677a00bdf8b2e1e2201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdf123505e4677a00bdf8b2e1e2201" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 36905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 } health_report { overall_health: UNKNOWN } } }
I20260929 18:41:00.066012 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.062457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.11.118.194:36905)
0929 18:41:00.062575 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:00.062581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:00.062596 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:00.062774 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:00.062777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:00.062834 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:00.062951 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:00.063190 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.063712 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.064556 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:00.064786 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:00.064900 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:00.065173 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:00.065609 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:00.065739 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:00.065802 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 18:41:00.065868 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:41:00.069016 18763 raft_consensus.cc:1261] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Refusing update from remote peer 71bdf123505e4677a00bdf8b2e1e2201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:00.069129 19036 raft_consensus.cc:1261] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Refusing update from remote peer 71bdf123505e4677a00bdf8b2e1e2201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:00.069545 19190 consensus_queue.cc:1035] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f54790de3458470ba4da284f400e5bc8" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 42325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:00.069754 19197 consensus_queue.cc:1035] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2ba5bf0ab84dd190fe8ffe19db51ea" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:00.071820 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.071464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:52763)
0929 18:41:00.071675 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.071744 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 18:41:00.073529 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.073100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:47943)
0929 18:41:00.073377 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.073445 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260929 18:41:00.076819 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.076501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.193:42341)
0929 18:41:00.076682 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.076742 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260929 18:41:00.077047 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.076101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.195:35357)
0929 18:41:00.076924 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.076975 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":743}
I20260929 18:41:00.080833 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:00.080589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39579 (local address 127.11.118.194:39929)
0929 18:41:00.080716 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.080770 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 18:41:00.083562 18624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:00.083724 18624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:00.087761 18624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa3d239fdefa4ba89ec98265eedf0a0f: Sending DeleteTablet for 3 replicas of tablet 6de3f917b66a4dc7ab41a9e84375a701
I20260929 18:41:00.088415 18880 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:00 UTC) from {username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:59594
I20260929 18:41:00.088685 18743 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:00 UTC) from {username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:34662
I20260929 18:41:00.088837 19211 tablet_replica.cc:330] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: stopping tablet replica
I20260929 18:41:00.088966 19211 raft_consensus.cc:2229] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:41:00.088965 19016 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de3f917b66a4dc7ab41a9e84375a701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:00 UTC) from {username='oryx', principal='oryx/127.11.118.252@KRBTEST.COM'} at 127.0.0.1:44090
I20260929 18:41:00.089164 19212 tablet_replica.cc:330] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8: stopping tablet replica
I20260929 18:41:00.089299 19212 raft_consensus.cc:2229] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:00.089432 19211 raft_consensus.cc:2258] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:00.089501 19212 raft_consensus.cc:2258] T 6de3f917b66a4dc7ab41a9e84375a701 P f54790de3458470ba4da284f400e5bc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:00.090020 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18590
I20260929 18:41:00.091312 19211 ts_tablet_manager.cc:1905] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:00.094478 19213 tablet_replica.cc:330] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: stopping tablet replica
I20260929 18:41:00.094710 19213 raft_consensus.cc:2229] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:00.094938 19213 raft_consensus.cc:2258] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:00.094973 19211 ts_tablet_manager.cc:1918] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:00.095085 19211 log.cc:1196] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/6de3f917b66a4dc7ab41a9e84375a701
I20260929 18:41:00.095479 19211 ts_tablet_manager.cc:1939] T 6de3f917b66a4dc7ab41a9e84375a701 P 71bdf123505e4677a00bdf8b2e1e2201: Deleting consensus metadata
I20260929 18:41:00.095813 19213 ts_tablet_manager.cc:1905] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:00.096304 18616 catalog_manager.cc:4841] TS 71bdf123505e4677a00bdf8b2e1e2201 (127.11.118.194:36905): tablet 6de3f917b66a4dc7ab41a9e84375a701 (table test-table [id=8fef290cf003449dbd69b08ad344da99]) successfully deleted
I20260929 18:41:00.098016 19213 ts_tablet_manager.cc:1918] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:00.098115 19213 log.cc:1196] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6de3f917b66a4dc7ab41a9e84375a701
I20260929 18:41:00.098502 19213 ts_tablet_manager.cc:1939] T 6de3f917b66a4dc7ab41a9e84375a701 P 2a2ba5bf0ab84dd190fe8ffe19db51ea: Deleting consensus metadata
I20260929 18:41:00.099234 18615 catalog_manager.cc:4841] TS 2a2ba5bf0ab84dd190fe8ffe19db51ea (127.11.118.195:46495): tablet 6de3f917b66a4dc7ab41a9e84375a701 (table test-table [id=8fef290cf003449dbd69b08ad344da99]) successfully deleted
W20260929 18:41:00.099458 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:00.098675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:42325 (local address 127.0.0.1:34672)
0929 18:41:00.098818 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:00.098835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:00.098931 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:00.099348 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:42325: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:41:00.099650 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18817
W20260929 18:41:00.099645 18615 catalog_manager.cc:4584] TS f54790de3458470ba4da284f400e5bc8 (127.11.118.193:42325): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6de3f917b66a4dc7ab41a9e84375a701: Network error: Client connection negotiation failed: client connection to 127.11.118.193:42325: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 18:41:00.105839 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18953
I20260929 18:41:00.113713 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18308
I20260929 18:41:00.120055 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 18409
2026-09-29T18:41:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7501 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:41:00 dist-test-slave-cgv7 krb5kdc[19216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:41:00 dist-test-slave-cgv7 krb5kdc[19216](info): set up 2 sockets
Sep 29 18:41:00 dist-test-slave-cgv7 krb5kdc[19216](info): commencing operation
krb5kdc: starting...
W20260929 18:41:01.175011 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:41:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab.
Entry for principal oryx/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab.
I20260929 18:41:01.301651 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37517
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab
--principal=oryx/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:01.412497 19229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:01.412832 19229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:01.412915 19229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:01.416967 19229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:01.417059 19229 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 18:41:01.417088 19229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:01.417132 19229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:01.417169 19229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:01.417204 19229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:01.422613 19229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:37517
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.254.keytab
--principal=oryx/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:01.423760 19229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:01.424813 19229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:41:01.433678 19229 server_base.cc:1034] running on GCE node
W20260929 18:41:01.433640 19234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:01.433660 19235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:01.433640 19237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:01.434484 19229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:01.435613 19229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:01.436808 19229 hybrid_clock.cc:648] HybridClock initialized: now 1790707261436788 us; error 34 us; skew 500 ppm
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:01.439574 19229 init.cc:377] Logged in from keytab as oryx/127.11.118.254@KRBTEST.COM (short username oryx)
I20260929 18:41:01.440454 19229 webserver.cc:460] Webserver started at http://127.11.118.254:34363/ using document root <none> and password file <none>
I20260929 18:41:01.441022 19229 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:01.441116 19229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:01.441349 19229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:01.443431 19229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "3a67a6683e4f46f39d6fb36b5fcaba71"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "9fa53589200fcc26be100be4d246e837"
server_key_iv: "8db0a05824c9b594fc7d2cc9f5930768"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.443998 19229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "3a67a6683e4f46f39d6fb36b5fcaba71"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "9fa53589200fcc26be100be4d246e837"
server_key_iv: "8db0a05824c9b594fc7d2cc9f5930768"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.447814 19229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:41:01.450245 19244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:01.451593 19229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:41:01.451766 19229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "3a67a6683e4f46f39d6fb36b5fcaba71"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "9fa53589200fcc26be100be4d246e837"
server_key_iv: "8db0a05824c9b594fc7d2cc9f5930768"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.451890 19229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:01.479249 19229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:01.482568 19229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:01.482792 19229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:01.491274 19229 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:37517
I20260929 18:41:01.491299 19306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:37517 every 8 connection(s)
I20260929 18:41:01.492250 19229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:01.495066 19307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:01.498449 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 19229
I20260929 18:41:01.498589 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:41:01.498879 11739 external_mini_cluster.cc:1428] Setting key b58f1fa30a25e60c943a21cef86cc21d
I20260929 18:41:01.501128 19307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Bootstrap starting.
I20260929 18:41:01.503481 19307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:01.504242 19307 log.cc:824] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:01.506335 19307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: No bootstrap required, opened a new log
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:01.510924 19307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } }
I20260929 18:41:01.511267 19307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:01.511353 19307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a67a6683e4f46f39d6fb36b5fcaba71, State: Initialized, Role: FOLLOWER
I20260929 18:41:01.511962 19307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } }
I20260929 18:41:01.512095 19307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:01.512187 19307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:01.512285 19307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:01.513455 19307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } }
I20260929 18:41:01.513832 19307 leader_election.cc:302] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a67a6683e4f46f39d6fb36b5fcaba71; no voters:
I20260929 18:41:01.514110 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.500338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.11.118.254:37517)
0929 18:41:01.500800 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:01.500816 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:01.500852 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:41:01.501534 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:01.501552 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:01.501894 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:01.502243 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:01.503122 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.504198 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.505341 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.505825 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.506001 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.508861 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:01.508866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:01.508882 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:01.508911 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:01.510739 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.511303 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.511315 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.511320 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.511516 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.511829 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.511831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.511832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.512364 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:01.512559 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:01.512876 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 18:41:01.513194 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":140,"threads_started":1}
I20260929 18:41:01.514178 19307 leader_election.cc:288] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:41:01.514339 19312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:01.514542 19312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [term 1 LEADER]: Becoming Leader. State: Replica: 3a67a6683e4f46f39d6fb36b5fcaba71, State: Running, Role: LEADER
I20260929 18:41:01.514871 19312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } }
I20260929 18:41:01.515637 19307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:01.517660 19313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a67a6683e4f46f39d6fb36b5fcaba71. Latest consensus state: current_term: 1 leader_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } } }
I20260929 18:41:01.517812 19313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:01.518023 19314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a67a6683e4f46f39d6fb36b5fcaba71" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37517 } } }
I20260929 18:41:01.518110 19314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:01.518200 19327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:01.521070 19327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:01.526998 19327 catalog_manager.cc:1347] Generated new cluster ID: dcc6ac4ea1b247b0baaafcb0b306e269
I20260929 18:41:01.527083 19327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:01.535632 19327 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:41:01.536595 19327 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:01.545885 19327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Generated new TSK 0
I20260929 18:41:01.546744 19327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
I20260929 18:41:01.577215 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37517
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:01.692695 19333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:01.692963 19333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:01.693041 19333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:01.697129 19333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:01.697190 19333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:01.697232 19333 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:41:01.697317 19333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:01.702716 19333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:01.703775 19333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:01.704785 19333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:01.714377 19341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:01.714408 19338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:01.714612 19333 server_base.cc:1034] running on GCE node
W20260929 18:41:01.714654 19339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:01.715135 19333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:01.715767 19333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:01.716974 19333 hybrid_clock.cc:648] HybridClock initialized: now 1790707261716939 us; error 47 us; skew 500 ppm
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:01.719733 19333 init.cc:377] Logged in from keytab as oryx/127.11.118.193@KRBTEST.COM (short username oryx)
I20260929 18:41:01.720595 19333 webserver.cc:460] Webserver started at http://127.11.118.193:37927/ using document root <none> and password file <none>
I20260929 18:41:01.721161 19333 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:01.721313 19333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:01.721546 19333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:01.723706 19333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "ca6f02aacf8b4bbc84697deb7f723101"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "d2da0543d32d10042d006327b1f50415"
server_key_iv: "fe24a5aed7e25ca912270a372ffb4d0e"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.724287 19333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "ca6f02aacf8b4bbc84697deb7f723101"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "d2da0543d32d10042d006327b1f50415"
server_key_iv: "fe24a5aed7e25ca912270a372ffb4d0e"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.728066 19333 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 18:41:01.730616 19348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:01.732018 19333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:41:01.732111 19333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "ca6f02aacf8b4bbc84697deb7f723101"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "d2da0543d32d10042d006327b1f50415"
server_key_iv: "fe24a5aed7e25ca912270a372ffb4d0e"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.732199 19333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:01.743541 19333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:01.746603 19333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:01.746768 19333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:01.748063 19333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:01.748137 19333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:01.748208 19333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:01.748243 19333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:01.759052 19333 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45155
I20260929 18:41:01.759171 19463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45155 every 8 connection(s)
I20260929 18:41:01.760215 19333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:01.762799 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.749010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:47049 (local address 127.11.118.254:37517)
0929 18:41:01.749142 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:01.749146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:01.750169 (+ 1023us) server_negotiation.cc:407] Connection header received
0929 18:41:01.751362 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:01.751368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:01.751433 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:01.751536 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:01.753087 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.754008 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.755624 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.755917 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.756056 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.758811 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:01.758817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:01.758821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:01.758853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:01.760471 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.761283 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.761286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.761288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.761373 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.761688 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.761697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.761699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.761949 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:01.762044 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:01.762554 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 18:41:01.762656 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:41:01.763510 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 19333
I20260929 18:41:01.763516 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.749511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.193:47049)
0929 18:41:01.750014 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:01.750055 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:01.751134 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:01.751730 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:01.751741 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:01.752204 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:01.752938 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.752960 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.754187 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.754190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.755500 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.755510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.756117 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.756121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.756366 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.757078 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:01.757105 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:01.758603 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:01.760617 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:01.760623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:01.760638 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:01.760984 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:01.761461 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:01.761463 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:01.761465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:01.761591 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:01.762048 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:01.762052 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:01.762317 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 18:41:01.762484 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 18:41:01.762722 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":137,"threads_started":1}
I20260929 18:41:01.763633 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:41:01.763953 11739 external_mini_cluster.cc:1428] Setting key f8f02f69f9073a2e072a490d9bdf2e3f
I20260929 18:41:01.765017 19464 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37517
I20260929 18:41:01.765192 19464 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:01.765776 19464 heartbeater.cc:511] Master 127.11.118.254:37517 requested a full tablet report, sending...
I20260929 18:41:01.767545 19256 ts_manager.cc:194] Registered new tserver with Master: ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155)
I20260929 18:41:01.769085 19256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:47049
WARNING: no policy specified for oryx/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:41:01.775000 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.767429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.193:39489)
0929 18:41:01.767807 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:01.767822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:01.767918 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:01.768194 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:01.768197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:01.768410 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:01.768528 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.768536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.769308 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.769312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.770044 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.770053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.770677 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.770680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.770749 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.771295 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:01.771307 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:01.771628 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:01.773370 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:01.773374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:01.773376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:01.773579 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:01.773865 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:01.773870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:01.773872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:01.773958 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:01.774433 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:01.774435 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:01.774536 (+ 101us) client_negotiation.cc:769] Sending connection context
0929 18:41:01.774646 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 18:41:01.774804 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":145,"threads_started":1}
I20260929 18:41:01.774979 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.767508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39489 (local address 127.11.118.254:37517)
0929 18:41:01.767635 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:01.767641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:01.767899 (+ 258us) server_negotiation.cc:407] Connection header received
0929 18:41:01.768009 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:01.768013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:01.768066 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:01.768165 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:01.768643 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.769177 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.770205 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.770493 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.770554 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.771772 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:01.771776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:01.771779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:01.771802 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:01.773230 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.773684 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.773688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.773690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.773770 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:01.774057 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:01.774062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:01.774063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:01.774329 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:01.774443 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:01.774698 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 18:41:01.774803 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
I20260929 18:41:01.791486 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37517
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:01.921509 19473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:01.921787 19473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:01.921833 19473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:01.926065 19473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:01.926148 19473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:01.926188 19473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:41:01.926293 19473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:01.931842 19473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:01.932999 19473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:01.934087 19473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:01.943912 19479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:01.943914 19478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:01.943912 19481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:01.944361 19473 server_base.cc:1034] running on GCE node
I20260929 18:41:01.944805 19473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:01.945446 19473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:01.946601 19473 hybrid_clock.cc:648] HybridClock initialized: now 1790707261946578 us; error 36 us; skew 500 ppm
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:01.949338 19473 init.cc:377] Logged in from keytab as oryx/127.11.118.194@KRBTEST.COM (short username oryx)
I20260929 18:41:01.950210 19473 webserver.cc:460] Webserver started at http://127.11.118.194:38447/ using document root <none> and password file <none>
I20260929 18:41:01.950799 19473 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:01.950892 19473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:01.951126 19473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:01.953204 19473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "ca663fed837e414db85dca97935e3a13"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "106a0f4afa88a432160bb778fedf3571"
server_key_iv: "43078b769674f511294484d8d1ed35cb"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.953791 19473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "ca663fed837e414db85dca97935e3a13"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "106a0f4afa88a432160bb778fedf3571"
server_key_iv: "43078b769674f511294484d8d1ed35cb"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.957767 19473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:41:01.960542 19488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:01.962038 19473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:01.962194 19473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "ca663fed837e414db85dca97935e3a13"
format_stamp: "Formatted at 2026-09-29 18:41:01 on dist-test-slave-cgv7"
server_key: "106a0f4afa88a432160bb778fedf3571"
server_key_iv: "43078b769674f511294484d8d1ed35cb"
server_key_version: "encryptionkey@0"
I20260929 18:41:01.962376 19473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:01.984745 19473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:01.987883 19473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:01.988113 19473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:01.989629 19473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:01.989704 19473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:01.989773 19473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:01.989816 19473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:41:01 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:02.000969 19473 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:40533
I20260929 18:41:02.001051 19603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:40533 every 8 connection(s)
I20260929 18:41:02.002099 19473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:02.004606 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.990466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:49901 (local address 127.11.118.254:37517)
0929 18:41:01.990596 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:01.990600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:01.991661 (+ 1061us) server_negotiation.cc:407] Connection header received
0929 18:41:01.992708 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:01.992713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:01.992882 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:01.992966 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:01.994747 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.995425 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.996726 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.997137 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.997271 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.000822 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.000826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.000828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.000856 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.002466 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.003129 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.003132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.003133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.003213 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.003586 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.003588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.003589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.003767 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.003844 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.004374 (+ 530us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.004481 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:41:02.005484 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:01.991015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.194:49901)
0929 18:41:01.991501 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:01.991544 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:01.992432 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:01.993510 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:01.993519 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:01.993940 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:01.994540 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.994559 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.995559 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.995563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.996534 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:01.996544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:01.997334 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:01.997339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:01.997560 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:01.998854 (+ 1294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:01.998871 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:02.000576 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:02.002669 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.002674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.002689 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.003013 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.003283 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.003286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.003290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.003455 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.003867 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:02.003872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:02.004124 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.004274 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.004522 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260929 18:41:02.006812 19604 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37517
I20260929 18:41:02.006973 19604 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:02.007490 19604 heartbeater.cc:511] Master 127.11.118.254:37517 requested a full tablet report, sending...
I20260929 18:41:02.008199 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 19473
I20260929 18:41:02.008342 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:41:02.008590 19256 ts_manager.cc:194] Registered new tserver with Master: ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533)
I20260929 18:41:02.008671 11739 external_mini_cluster.cc:1428] Setting key 3a402560d0a28e183c219d52d4f51f5b
I20260929 18:41:02.009380 19256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:49901
I20260929 18:41:02.017740 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.009058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:53643 (local address 127.11.118.254:37517)
0929 18:41:02.009188 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.009193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.009453 (+ 260us) server_negotiation.cc:407] Connection header received
0929 18:41:02.009564 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.009567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.009619 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.009706 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.010207 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.010772 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.012043 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.012425 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.012560 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.013817 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.013824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.013830 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.013863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.015780 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.016213 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.016218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.016220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.016333 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.016755 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.016759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.016761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.016992 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.017075 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.017431 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.017614 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:41:02.017779 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.008964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.194:53643)
0929 18:41:02.009355 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.009370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.009473 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.009725 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.009728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.009961 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:02.010103 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.010112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.010915 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.010921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.011914 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.011924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.012562 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.012566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.012653 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.013230 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:02.013245 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:02.013659 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:02.015890 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.015893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.015894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.016104 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.016497 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.016503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.016519 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.016628 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.017102 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:02.017108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:02.017236 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.017382 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.017555 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
I20260929 18:41:02.038347 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:37517
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:02.152598 19613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:02.152915 19613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:02.153000 19613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:02.157249 19613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:02.157346 19613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:02.157414 19613 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 18:41:02.157500 19613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:02.163002 19613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.118.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:02.164135 19613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:02.165252 19613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:41:02.175129 19613 server_base.cc:1034] running on GCE node
W20260929 18:41:02.175061 19618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:02.175061 19621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:02.175083 19619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:02.175766 19613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:02.176455 19613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:02.177656 19613 hybrid_clock.cc:648] HybridClock initialized: now 1790707262177623 us; error 51 us; skew 500 ppm
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707262, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:02.180593 19613 init.cc:377] Logged in from keytab as oryx/127.11.118.195@KRBTEST.COM (short username oryx)
I20260929 18:41:02.181463 19613 webserver.cc:460] Webserver started at http://127.11.118.195:34961/ using document root <none> and password file <none>
I20260929 18:41:02.182034 19613 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:02.182132 19613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:02.182422 19613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:02.184603 19613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "44b473ac8a0b4880bead2b3eb75760b9"
format_stamp: "Formatted at 2026-09-29 18:41:02 on dist-test-slave-cgv7"
server_key: "a5162e70960de31bb554c5a9ace22295"
server_key_iv: "29dfb021632754c606aff12560f6bb78"
server_key_version: "encryptionkey@0"
I20260929 18:41:02.185168 19613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "44b473ac8a0b4880bead2b3eb75760b9"
format_stamp: "Formatted at 2026-09-29 18:41:02 on dist-test-slave-cgv7"
server_key: "a5162e70960de31bb554c5a9ace22295"
server_key_iv: "29dfb021632754c606aff12560f6bb78"
server_key_version: "encryptionkey@0"
I20260929 18:41:02.188917 19613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 18:41:02.191511 19628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:02.192869 19613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 18:41:02.193018 19613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "44b473ac8a0b4880bead2b3eb75760b9"
format_stamp: "Formatted at 2026-09-29 18:41:02 on dist-test-slave-cgv7"
server_key: "a5162e70960de31bb554c5a9ace22295"
server_key_iv: "29dfb021632754c606aff12560f6bb78"
server_key_version: "encryptionkey@0"
I20260929 18:41:02.193143 19613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:02.219468 19613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:02.223075 19613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:02.223308 19613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:02.224880 19613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:02.224961 19613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:02.225025 19613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:02.225071 19613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707262, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:02.236409 19613 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:45487
I20260929 18:41:02.237511 19613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:02.238404 19743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:45487 every 8 connection(s)
I20260929 18:41:02.240926 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.226040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:58483 (local address 127.11.118.254:37517)
0929 18:41:02.226192 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.226196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.227404 (+ 1208us) server_negotiation.cc:407] Connection header received
0929 18:41:02.228752 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.228756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.228819 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.228913 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.230852 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.231749 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.233167 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.233523 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.233659 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.237053 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.237057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.237059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.237087 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.238958 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.239459 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.239462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.239463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.239547 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.239911 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.239914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.239915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.240110 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.240188 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.240666 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.240803 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 18:41:02.241822 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.226692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.195:58483)
0929 18:41:02.227238 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.227285 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.228503 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.229149 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.229162 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.229741 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:02.230611 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.230641 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.231955 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.231959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.233039 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.233050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.233704 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.233708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.233871 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.234681 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:02.234702 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:02.236850 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:02.239075 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.239080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.239092 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.239348 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.239661 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.239663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.239665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.239798 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.240199 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:02.240204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:02.240448 (+ 244us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.240598 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.240901 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":152,"threads_started":1}
I20260929 18:41:02.243234 19744 heartbeater.cc:347] Connected to a master server at 127.11.118.254:37517
I20260929 18:41:02.243465 19744 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:02.244076 19744 heartbeater.cc:511] Master 127.11.118.254:37517 requested a full tablet report, sending...
I20260929 18:41:02.245216 19256 ts_manager.cc:194] Registered new tserver with Master: 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:02.245405 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 19613
I20260929 18:41:02.245540 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:41:02.245841 11739 external_mini_cluster.cc:1428] Setting key 8f3c045abc27c9319f7eef8386c808bf
I20260929 18:41:02.246026 19256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:58483
I20260929 18:41:02.247848 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:41:02.253463 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.245471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.11.118.195:55539)
0929 18:41:02.245797 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.245811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.245906 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.246420 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.246423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.246670 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:02.246810 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.246817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.247707 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.247710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.248319 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.248328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.248789 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.248792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.248888 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.249444 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:02.249455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:02.249945 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:02.251600 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.251607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.251612 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.251900 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.252223 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.252230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.252235 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.252347 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.252809 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:02.252812 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:02.252929 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.253049 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.253206 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260929 18:41:02.253410 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.245452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:55539 (local address 127.11.118.254:37517)
0929 18:41:02.245589 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.245593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.245898 (+ 305us) server_negotiation.cc:407] Connection header received
0929 18:41:02.245993 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.245996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.246044 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.246145 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.246921 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.247476 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.248451 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.248677 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.248793 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.250078 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.250081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.250083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.250111 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.251494 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.252024 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.252027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.252029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.252118 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.252476 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.252480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.252482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.252695 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.252778 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.253096 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.253195 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 18:41:02.273352 19310 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:02.267002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.11.118.254:37517)
0929 18:41:02.267147 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.267150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.267163 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:02.267203 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.267207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.267247 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.267342 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.268098 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.268627 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.269512 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.269746 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.269855 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.273231 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47214: BlockingRecv error: recv got EOF from 127.0.0.1:47214 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:02.283334 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.275129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.11.118.254:37517)
0929 18:41:02.275277 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.275281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.275469 (+ 188us) server_negotiation.cc:407] Connection header received
0929 18:41:02.275580 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.275582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.275624 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.275694 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.276294 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.276797 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.277559 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.277811 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.277889 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.280262 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.280265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.280267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.280291 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.281751 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.282121 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.282123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.282125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.282194 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.282479 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.282481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.282482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.282650 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.282778 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.283045 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.283140 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 18:41:02.286551 19256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:41:02.289605 19256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:02.306794 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.300659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.11.118.194:40533)
0929 18:41:02.301109 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.301123 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.301145 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:41:02.301220 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.301228 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.301425 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.301589 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.301864 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.302963 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.305165 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.305988 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306153 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.306352 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.306542 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.306639 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":119,"threads_started":1}
I20260929 18:41:02.306785 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.300503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:40533 (local address 127.0.0.1:34152)
0929 18:41:02.300762 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.300775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.300865 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.301586 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.301589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.301605 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.301717 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.301724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.303092 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.303096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.305032 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.305041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.306119 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.306284 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.306356 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.306509 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.306590 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":812,"thread_start_us":109,"threads_started":1}
I20260929 18:41:02.306800 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.299762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45155 (local address 127.0.0.1:49004)
0929 18:41:02.300165 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.300221 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.300367 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.300970 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.300977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.300998 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.301147 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.301154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.302705 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.302712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.303842 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.303850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.306156 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.306313 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.306363 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.306452 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.306590 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":28,"thread_start_us":100,"threads_started":1}
I20260929 18:41:02.307212 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.300833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:45487 (local address 127.0.0.1:57988)
0929 18:41:02.302542 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.302555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.302654 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.303073 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.303076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.303094 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.303254 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.303261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.304143 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.304147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.305226 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.305234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.306790 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.306898 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.306923 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.306985 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.307047 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1651,"mutex_wait_us":26,"thread_start_us":74,"threads_started":1}
I20260929 18:41:02.307403 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.299915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.11.118.193:45155)
0929 18:41:02.300205 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.300212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.300299 (+ 87us) server_negotiation.cc:407] Connection header received
0929 18:41:02.300535 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.300541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.300789 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.301304 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.301440 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.302583 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.305189 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.306006 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306978 (+ 972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.307105 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.307161 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.307263 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":66,"threads_started":1}
I20260929 18:41:02.307715 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.301185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.11.118.195:45487)
0929 18:41:02.301476 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.301486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.302656 (+ 1170us) server_negotiation.cc:407] Connection header received
0929 18:41:02.302760 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.302764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.302929 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.303083 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.303360 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.304035 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.305363 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.306462 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.306584 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.306705 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.307492 (+ 787us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.307594 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":111,"threads_started":1}
I20260929 18:41:02.310583 19538 tablet_service.cc:1467] Processing CreateTablet for tablet 71f81d9b26c1446aa2019e46a403265b (DEFAULT_TABLE table=test-table [id=06cb5f5be0664dc68c8fb5066aad4286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.310571 19678 tablet_service.cc:1467] Processing CreateTablet for tablet 71f81d9b26c1446aa2019e46a403265b (DEFAULT_TABLE table=test-table [id=06cb5f5be0664dc68c8fb5066aad4286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.310567 19398 tablet_service.cc:1467] Processing CreateTablet for tablet 71f81d9b26c1446aa2019e46a403265b (DEFAULT_TABLE table=test-table [id=06cb5f5be0664dc68c8fb5066aad4286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.311648 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f81d9b26c1446aa2019e46a403265b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.311666 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f81d9b26c1446aa2019e46a403265b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.311659 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f81d9b26c1446aa2019e46a403265b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.319067 19772 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Bootstrap starting.
I20260929 18:41:02.319988 19774 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Bootstrap starting.
I20260929 18:41:02.320123 19773 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Bootstrap starting.
I20260929 18:41:02.322006 19774 tablet_bootstrap.cc:654] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.322201 19773 tablet_bootstrap.cc:654] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.322202 19772 tablet_bootstrap.cc:654] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.322876 19774 log.cc:824] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:02.323040 19772 log.cc:824] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:02.323040 19773 log.cc:824] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:02.324740 19774 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: No bootstrap required, opened a new log
I20260929 18:41:02.324740 19773 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: No bootstrap required, opened a new log
I20260929 18:41:02.324954 19774 ts_tablet_manager.cc:1397] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260929 18:41:02.324960 19773 ts_tablet_manager.cc:1397] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 18:41:02.327819 19772 tablet_bootstrap.cc:492] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: No bootstrap required, opened a new log
I20260929 18:41:02.328038 19772 ts_tablet_manager.cc:1397] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260929 18:41:02.328531 19774 raft_consensus.cc:348] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.328795 19774 raft_consensus.cc:374] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.328869 19774 raft_consensus.cc:729] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.329344 19774 consensus_queue.cc:260] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.329205 19773 raft_consensus.cc:348] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.329483 19773 raft_consensus.cc:374] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.329586 19773 raft_consensus.cc:729] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.330010 19773 consensus_queue.cc:260] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.330444 19774 ts_tablet_manager.cc:1428] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260929 18:41:02.330710 19464 heartbeater.cc:503] Master 127.11.118.254:37517 was elected leader, sending a full tablet report...
I20260929 18:41:02.330796 19744 heartbeater.cc:503] Master 127.11.118.254:37517 was elected leader, sending a full tablet report...
I20260929 18:41:02.330978 19773 ts_tablet_manager.cc:1428] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260929 18:41:02.332821 19772 raft_consensus.cc:348] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.333127 19772 raft_consensus.cc:374] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.333186 19772 raft_consensus.cc:729] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca663fed837e414db85dca97935e3a13, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.333633 19772 consensus_queue.cc:260] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.334690 19604 heartbeater.cc:503] Master 127.11.118.254:37517 was elected leader, sending a full tablet report...
I20260929 18:41:02.334699 19772 ts_tablet_manager.cc:1428] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 18:41:02.424502 19779 raft_consensus.cc:482] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:02.424764 19779 raft_consensus.cc:504] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.425904 19779 leader_election.cc:288] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:02.430303 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.426360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:46905 (local address 127.11.118.194:40533)
0929 18:41:02.426490 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.426494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.426623 (+ 129us) server_negotiation.cc:407] Connection header received
0929 18:41:02.426834 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.426838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.426897 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.426986 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.427408 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.428026 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429278 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.429688 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429812 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.429871 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.430064 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.430143 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:41:02.430336 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.426800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39461 (local address 127.11.118.195:45487)
0929 18:41:02.426960 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.426965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.427117 (+ 152us) server_negotiation.cc:407] Connection header received
0929 18:41:02.427258 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.427261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.427316 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.427392 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.427701 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.428260 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429316 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.429717 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429855 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.429910 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.430125 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.430202 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 18:41:02.430322 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.426206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:40533 (local address 127.11.118.193:46905)
0929 18:41:02.426520 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.426537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.426653 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.427030 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.427032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.427049 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.427259 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.427266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.428161 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.428165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.429143 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.429153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.429822 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.429915 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.429947 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.430020 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.430115 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"spinlock_wait_cycles":33792,"thread_start_us":146,"threads_started":1}
I20260929 18:41:02.430344 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.426715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:45487 (local address 127.11.118.193:39461)
0929 18:41:02.427017 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.427030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.427144 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.427416 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.427418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.427435 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.427545 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.427550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.428373 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.428376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.429161 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.429170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.429867 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.429870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.429976 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.429993 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.430090 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.430151 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"spinlock_wait_cycles":1280,"thread_start_us":104,"threads_started":1}
I20260929 18:41:02.431056 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f81d9b26c1446aa2019e46a403265b" candidate_uuid: "ca6f02aacf8b4bbc84697deb7f723101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca663fed837e414db85dca97935e3a13" is_pre_election: true
I20260929 18:41:02.431061 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f81d9b26c1446aa2019e46a403265b" candidate_uuid: "ca6f02aacf8b4bbc84697deb7f723101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9" is_pre_election: true
I20260929 18:41:02.431365 19557 raft_consensus.cc:2454] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6f02aacf8b4bbc84697deb7f723101 in term 0.
I20260929 18:41:02.431367 19683 raft_consensus.cc:2454] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6f02aacf8b4bbc84697deb7f723101 in term 0.
I20260929 18:41:02.431865 19352 leader_election.cc:302] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b473ac8a0b4880bead2b3eb75760b9, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:02.432189 19779 raft_consensus.cc:2781] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:02.432282 19779 raft_consensus.cc:482] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:02.432317 19779 raft_consensus.cc:3037] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.433450 19779 raft_consensus.cc:504] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.433805 19779 leader_election.cc:288] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [CANDIDATE]: Term 1 election: Requested vote from peers ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:02.434330 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f81d9b26c1446aa2019e46a403265b" candidate_uuid: "ca6f02aacf8b4bbc84697deb7f723101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9"
I20260929 18:41:02.434331 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f81d9b26c1446aa2019e46a403265b" candidate_uuid: "ca6f02aacf8b4bbc84697deb7f723101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca663fed837e414db85dca97935e3a13"
I20260929 18:41:02.434481 19557 raft_consensus.cc:3037] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.434455 19683 raft_consensus.cc:3037] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.435750 19557 raft_consensus.cc:2454] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6f02aacf8b4bbc84697deb7f723101 in term 1.
I20260929 18:41:02.435891 19683 raft_consensus.cc:2454] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6f02aacf8b4bbc84697deb7f723101 in term 1.
I20260929 18:41:02.436110 19352 leader_election.cc:302] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:02.436298 19779 raft_consensus.cc:2781] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:02.436542 19779 raft_consensus.cc:686] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 1 LEADER]: Becoming Leader. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Running, Role: LEADER
I20260929 18:41:02.436877 19779 consensus_queue.cc:237] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.440281 19256 catalog_manager.cc:5495] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193). New cstate: current_term: 1 leader_uuid: "ca6f02aacf8b4bbc84697deb7f723101" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } health_report { overall_health: UNKNOWN } } }
I20260929 18:41:02.457509 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.453809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.11.118.193:45155)
0929 18:41:02.453950 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.453954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.453968 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:02.454125 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.454128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.454183 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.454332 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:02.454577 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.455096 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.456051 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.456276 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.456388 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.456682 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:02.457112 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:02.457241 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.457307 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.457364 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:41:02.463310 19683 raft_consensus.cc:1261] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Refusing update from remote peer ca6f02aacf8b4bbc84697deb7f723101: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:02.463340 19557 raft_consensus.cc:1261] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Refusing update from remote peer ca6f02aacf8b4bbc84697deb7f723101: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:02.464026 19779 consensus_queue.cc:1035] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:02.464282 19783 consensus_queue.cc:1035] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:02.474743 19788 mvcc.cc:205] Tried to move back new op lower bound from 7334736947042430976 to 7334736946943954944. Current Snapshot: MvccSnapshot[applied={T|T < 7334736947042430976}]
I20260929 18:41:02.481220 19256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:02.481423 19256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:02.483845 19256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Sending DeleteTablet for 3 replicas of tablet 71f81d9b26c1446aa2019e46a403265b
I20260929 18:41:02.484741 19678 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f81d9b26c1446aa2019e46a403265b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:02 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57988
I20260929 18:41:02.484890 19538 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f81d9b26c1446aa2019e46a403265b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:02 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34152
I20260929 18:41:02.485276 19796 tablet_replica.cc:330] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: stopping tablet replica
I20260929 18:41:02.485528 19398 tablet_service.cc:1514] Processing DeleteTablet for tablet 71f81d9b26c1446aa2019e46a403265b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:02 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49004
I20260929 18:41:02.485939 19796 raft_consensus.cc:2229] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:02.485975 19798 tablet_replica.cc:330] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: stopping tablet replica
I20260929 18:41:02.485891 19256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 18:41:02.486227 19798 raft_consensus.cc:2229] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 1 LEADER]: Raft consensus shutting down.
W20260929 18:41:02.486348 19256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:02.486370 19796 raft_consensus.cc:2258] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:02.486575 19798 raft_consensus.cc:2258] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:02.486930 19797 tablet_replica.cc:330] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: stopping tablet replica
I20260929 18:41:02.487183 19797 raft_consensus.cc:2229] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:02.487444 19797 raft_consensus.cc:2258] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:02.487861 19796 ts_tablet_manager.cc:1905] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:02.489285 19798 ts_tablet_manager.cc:1905] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:02.492414 19798 ts_tablet_manager.cc:1918] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:02.492584 19798 log.cc:1196] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/71f81d9b26c1446aa2019e46a403265b
I20260929 18:41:02.492944 19798 ts_tablet_manager.cc:1939] T 71f81d9b26c1446aa2019e46a403265b P ca6f02aacf8b4bbc84697deb7f723101: Deleting consensus metadata
I20260929 18:41:02.494307 19248 catalog_manager.cc:4841] TS ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155): tablet 71f81d9b26c1446aa2019e46a403265b (table test-table [id=06cb5f5be0664dc68c8fb5066aad4286]) successfully deleted
I20260929 18:41:02.494927 19796 ts_tablet_manager.cc:1918] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:02.495062 19796 log.cc:1196] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/71f81d9b26c1446aa2019e46a403265b
I20260929 18:41:02.495370 19796 ts_tablet_manager.cc:1939] T 71f81d9b26c1446aa2019e46a403265b P 44b473ac8a0b4880bead2b3eb75760b9: Deleting consensus metadata
I20260929 18:41:02.496493 19248 catalog_manager.cc:4841] TS 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487): tablet 71f81d9b26c1446aa2019e46a403265b (table test-table [id=06cb5f5be0664dc68c8fb5066aad4286]) successfully deleted
I20260929 18:41:02.500015 19398 tablet_service.cc:1467] Processing CreateTablet for tablet b01aa15d686946bbbee80b098e419097 (DEFAULT_TABLE table=test-table [id=6d3f18c0a76e4058a279352bae871ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.500159 19797 ts_tablet_manager.cc:1905] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:02.500308 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01aa15d686946bbbee80b098e419097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.500321 19538 tablet_service.cc:1467] Processing CreateTablet for tablet b01aa15d686946bbbee80b098e419097 (DEFAULT_TABLE table=test-table [id=6d3f18c0a76e4058a279352bae871ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.500346 19678 tablet_service.cc:1467] Processing CreateTablet for tablet b01aa15d686946bbbee80b098e419097 (DEFAULT_TABLE table=test-table [id=6d3f18c0a76e4058a279352bae871ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:02.500545 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01aa15d686946bbbee80b098e419097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.500592 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01aa15d686946bbbee80b098e419097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.502863 19773 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Bootstrap starting.
I20260929 18:41:02.502862 19797 ts_tablet_manager.cc:1918] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:02.502952 19797 log.cc:1196] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/71f81d9b26c1446aa2019e46a403265b
I20260929 18:41:02.503185 19774 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Bootstrap starting.
I20260929 18:41:02.503278 19797 ts_tablet_manager.cc:1939] T 71f81d9b26c1446aa2019e46a403265b P ca663fed837e414db85dca97935e3a13: Deleting consensus metadata
I20260929 18:41:02.503859 19773 tablet_bootstrap.cc:654] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.504135 19774 tablet_bootstrap.cc:654] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.504375 19772 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Bootstrap starting.
I20260929 18:41:02.505287 19772 tablet_bootstrap.cc:654] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.505393 19774 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: No bootstrap required, opened a new log
I20260929 18:41:02.505458 19774 ts_tablet_manager.cc:1397] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:02.505874 19774 raft_consensus.cc:348] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.505990 19774 raft_consensus.cc:374] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.506021 19774 raft_consensus.cc:729] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.506158 19774 consensus_queue.cc:260] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.506217 19248 catalog_manager.cc:4841] TS ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533): tablet 71f81d9b26c1446aa2019e46a403265b (table test-table [id=06cb5f5be0664dc68c8fb5066aad4286]) successfully deleted
I20260929 18:41:02.507175 19772 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: No bootstrap required, opened a new log
I20260929 18:41:02.507262 19772 ts_tablet_manager.cc:1397] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:02.507431 19774 ts_tablet_manager.cc:1428] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260929 18:41:02.507721 19772 raft_consensus.cc:348] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.507834 19772 raft_consensus.cc:374] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.507876 19772 raft_consensus.cc:729] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca663fed837e414db85dca97935e3a13, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.507997 19772 consensus_queue.cc:260] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.508260 19772 ts_tablet_manager.cc:1428] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:02.509505 19773 tablet_bootstrap.cc:492] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: No bootstrap required, opened a new log
I20260929 18:41:02.509629 19773 ts_tablet_manager.cc:1397] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260929 18:41:02.510180 19773 raft_consensus.cc:348] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.510669 19773 raft_consensus.cc:374] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.510726 19773 raft_consensus.cc:729] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.510876 19773 consensus_queue.cc:260] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.511147 19773 ts_tablet_manager.cc:1428] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260929 18:41:02.512724 19465 tablet.cc:2369] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:41:02.738876 19745 tablet.cc:2369] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:41:02.755970 19605 tablet.cc:2369] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:41:02.806015 19780 raft_consensus.cc:482] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:02.806174 19780 raft_consensus.cc:504] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.807643 19780 leader_election.cc:288] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:02.813072 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.808418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:37967 (local address 127.11.118.195:45487)
0929 18:41:02.808555 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.808562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.808724 (+ 162us) server_negotiation.cc:407] Connection header received
0929 18:41:02.808816 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.808818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.808854 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.808914 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.809165 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.809914 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.811684 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.812313 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.812450 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.812504 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.812859 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.812940 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:41:02.813072 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.807967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:55175 (local address 127.11.118.193:45155)
0929 18:41:02.808160 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.808168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.808290 (+ 122us) server_negotiation.cc:407] Connection header received
0929 18:41:02.808472 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.808475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.808530 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.808612 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.808998 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.809826 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.811728 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.812313 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.812450 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.812504 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.812859 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.812940 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 18:41:02.813094 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.807877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45155 (local address 127.11.118.194:55175)
0929 18:41:02.808162 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.808177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.808342 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.808655 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.808661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.808689 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.808856 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.808863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.809938 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.809941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.811546 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.811558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.812551 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.812554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.812720 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.812736 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.812809 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.812907 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":213,"thread_start_us":100,"threads_started":1}
I20260929 18:41:02.813100 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.808298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:45487 (local address 127.11.118.194:37967)
0929 18:41:02.808621 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.808635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.808729 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.808947 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.808952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.808971 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.809055 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.809060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.810008 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.810011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.811546 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.811558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.812496 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.812500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.812686 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.812718 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.812809 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.812907 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":143,"thread_start_us":55,"threads_started":1}
I20260929 18:41:02.813521 19778 raft_consensus.cc:482] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:02.813743 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101" is_pre_election: true
I20260929 18:41:02.813651 19778 raft_consensus.cc:504] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.814018 19418 raft_consensus.cc:2454] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:02.814037 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9" is_pre_election: true
I20260929 18:41:02.814313 19683 raft_consensus.cc:2454] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:02.814520 19492 leader_election.cc:302] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:02.814813 19780 raft_consensus.cc:2781] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:02.814891 19780 raft_consensus.cc:482] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:02.814939 19780 raft_consensus.cc:3037] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.815140 19778 leader_election.cc:288] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533)
I20260929 18:41:02.816476 19780 raft_consensus.cc:504] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.816893 19780 leader_election.cc:288] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Requested vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:02.817327 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101"
I20260929 18:41:02.817449 19418 raft_consensus.cc:3037] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.817414 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9"
I20260929 18:41:02.817524 19683 raft_consensus.cc:3037] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.818774 19418 raft_consensus.cc:2454] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:02.818790 19683 raft_consensus.cc:2454] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:02.819116 19492 leader_election.cc:302] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:02.819398 19780 raft_consensus.cc:2781] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:02.819656 19780 raft_consensus.cc:686] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Becoming Leader. State: Replica: ca663fed837e414db85dca97935e3a13, State: Running, Role: LEADER
I20260929 18:41:02.819916 19780 consensus_queue.cc:237] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.820389 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.816074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:53551 (local address 127.11.118.193:45155)
0929 18:41:02.816201 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.816205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.816221 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:41:02.816263 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.816265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.816300 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.816421 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.816761 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.817528 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.819171 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.819708 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.819820 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.819869 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.820186 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.820271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:41:02.820405 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.815346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45155 (local address 127.11.118.195:53551)
0929 18:41:02.815672 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.815688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.815778 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.816434 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.816437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.816455 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.816577 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.816585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.817690 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.817693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.818999 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.819010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.819925 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.819929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.820038 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.820063 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.820145 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.820237 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260929 18:41:02.820892 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101" is_pre_election: true
I20260929 18:41:02.820996 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.815704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:40533 (local address 127.11.118.195:41387)
0929 18:41:02.816164 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.816178 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.816269 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.816878 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.816881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.816903 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:02.817069 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.817080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.818157 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.818160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.819392 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.819403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.820587 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.820592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.820693 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.820721 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.820795 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.820862 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":64,"threads_started":1}
I20260929 18:41:02.821069 19418 raft_consensus.cc:2379] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 44b473ac8a0b4880bead2b3eb75760b9 in current term 1: Already voted for candidate ca663fed837e414db85dca97935e3a13 in this term.
I20260929 18:41:02.821786 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.816110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:41387 (local address 127.11.118.194:40533)
0929 18:41:02.816577 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.816612 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.816627 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:02.816714 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.816717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.816772 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.816842 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:02.817199 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.817903 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.819559 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.820158 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.820258 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.820301 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.821620 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.821697 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389}
I20260929 18:41:02.822109 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01aa15d686946bbbee80b098e419097" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca663fed837e414db85dca97935e3a13" is_pre_election: true
I20260929 18:41:02.822418 19255 catalog_manager.cc:5495] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca663fed837e414db85dca97935e3a13 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "ca663fed837e414db85dca97935e3a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } health_report { overall_health: UNKNOWN } } }
I20260929 18:41:02.822602 19632 leader_election.cc:302] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44b473ac8a0b4880bead2b3eb75760b9; no voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101
I20260929 18:41:02.822866 19778 raft_consensus.cc:2726] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Sep 29 18:41:02 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707261, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.254@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:02.908358 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.900063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:37517 (local address 127.0.0.1:57562)
0929 18:41:02.900379 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:02.900397 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:02.900504 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:02.900739 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:02.900741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:02.901151 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:02.901251 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.901257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.902017 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.902020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.902636 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:02.902644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.903067 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.903070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:02.903131 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.903688 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:02.903702 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:02.905142 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:02.906898 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.906906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.906911 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.907136 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.907382 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:02.907385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:02.907386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:02.907462 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:02.907849 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:02.907855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:02.907961 (+ 106us) client_negotiation.cc:769] Sending connection context
0929 18:41:02.908055 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 18:41:02.908183 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":148,"threads_started":1}
I20260929 18:41:02.908391 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:02.900259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.11.118.254:37517)
0929 18:41:02.900512 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:02.900520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:02.900535 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:02.900574 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:02.900576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:02.900614 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:02.900702 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:02.901361 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.901906 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.902733 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:02.902968 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:02.903047 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:02.905281 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:02.905285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:02.905287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:02.905307 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:02.906798 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.907220 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.907223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.907224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.907301 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:02.907519 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:02.907521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:02.907522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:02.907730 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:02.907830 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:02.908114 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 18:41:02.908247 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":184,"thread_start_us":110,"threads_started":1}
I20260929 18:41:02.912413 19255 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57562:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 18:41:02.913101 19255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:02.917733 19398 tablet_service.cc:1467] Processing CreateTablet for tablet 3c61766453144beb84dfa84549ef265d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ccc7dc0db14fb8ac268fa933053d14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:41:02.917878 19678 tablet_service.cc:1467] Processing CreateTablet for tablet 3c61766453144beb84dfa84549ef265d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ccc7dc0db14fb8ac268fa933053d14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:41:02.917894 19538 tablet_service.cc:1467] Processing CreateTablet for tablet 3c61766453144beb84dfa84549ef265d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ccc7dc0db14fb8ac268fa933053d14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 18:41:02.918195 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c61766453144beb84dfa84549ef265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.918198 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c61766453144beb84dfa84549ef265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.918351 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c61766453144beb84dfa84549ef265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:02.920894 19772 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: Bootstrap starting.
I20260929 18:41:02.921118 19773 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: Bootstrap starting.
I20260929 18:41:02.921733 19774 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: Bootstrap starting.
I20260929 18:41:02.921941 19772 tablet_bootstrap.cc:654] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.921957 19773 tablet_bootstrap.cc:654] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.922751 19774 tablet_bootstrap.cc:654] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:02.923259 19772 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: No bootstrap required, opened a new log
I20260929 18:41:02.923357 19772 ts_tablet_manager.cc:1397] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 18:41:02.923828 19772 raft_consensus.cc:348] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.923918 19774 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: No bootstrap required, opened a new log
I20260929 18:41:02.923950 19772 raft_consensus.cc:374] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.923990 19774 ts_tablet_manager.cc:1397] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:02.923985 19772 raft_consensus.cc:729] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca663fed837e414db85dca97935e3a13, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.924140 19772 consensus_queue.cc:260] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.924427 19772 ts_tablet_manager.cc:1428] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:41:02.924409 19774 raft_consensus.cc:348] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.924389 19780 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:02.924512 19774 raft_consensus.cc:374] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.924544 19780 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:02.924554 19774 raft_consensus.cc:729] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.924675 19774 consensus_queue.cc:260] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.924991 19774 ts_tablet_manager.cc:1428] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:02.925240 19773 tablet_bootstrap.cc:492] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: No bootstrap required, opened a new log
I20260929 18:41:02.925331 19773 ts_tablet_manager.cc:1397] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260929 18:41:02.925540 19778 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:02.925658 19778 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:02.925762 19773 raft_consensus.cc:348] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.925874 19773 raft_consensus.cc:374] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:02.925913 19773 raft_consensus.cc:729] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Initialized, Role: FOLLOWER
I20260929 18:41:02.926040 19773 consensus_queue.cc:260] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.926406 19773 ts_tablet_manager.cc:1428] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:02.926558 19779 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:02.926685 19779 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:02.982450 19778 raft_consensus.cc:482] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:02.982654 19778 raft_consensus.cc:504] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.983122 19778 leader_election.cc:288] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533)
I20260929 18:41:02.983455 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c61766453144beb84dfa84549ef265d" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101" is_pre_election: true
I20260929 18:41:02.983650 19418 raft_consensus.cc:2454] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b473ac8a0b4880bead2b3eb75760b9 in term 0.
I20260929 18:41:02.983662 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c61766453144beb84dfa84549ef265d" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca663fed837e414db85dca97935e3a13" is_pre_election: true
I20260929 18:41:02.983819 19557 raft_consensus.cc:2454] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b473ac8a0b4880bead2b3eb75760b9 in term 0.
I20260929 18:41:02.984048 19632 leader_election.cc:302] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b473ac8a0b4880bead2b3eb75760b9, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:02.984370 19778 raft_consensus.cc:2781] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:02.984439 19778 raft_consensus.cc:482] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:02.984489 19778 raft_consensus.cc:3037] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.985579 19778 raft_consensus.cc:504] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.985867 19778 leader_election.cc:288] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 election: Requested vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533)
I20260929 18:41:02.986388 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c61766453144beb84dfa84549ef265d" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101"
I20260929 18:41:02.986479 19557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c61766453144beb84dfa84549ef265d" candidate_uuid: "44b473ac8a0b4880bead2b3eb75760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca663fed837e414db85dca97935e3a13"
I20260929 18:41:02.986562 19418 raft_consensus.cc:3037] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.986615 19557 raft_consensus.cc:3037] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:02.987592 19557 raft_consensus.cc:2454] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b473ac8a0b4880bead2b3eb75760b9 in term 1.
I20260929 18:41:02.987627 19418 raft_consensus.cc:2454] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b473ac8a0b4880bead2b3eb75760b9 in term 1.
I20260929 18:41:02.987917 19632 leader_election.cc:302] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b473ac8a0b4880bead2b3eb75760b9, ca663fed837e414db85dca97935e3a13; no voters:
I20260929 18:41:02.988123 19778 raft_consensus.cc:2781] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:02.988305 19778 raft_consensus.cc:686] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 LEADER]: Becoming Leader. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Running, Role: LEADER
I20260929 18:41:02.988579 19778 consensus_queue.cc:237] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:02.989964 19818 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: TxnStatusTablet state changed. Reason: New leader 44b473ac8a0b4880bead2b3eb75760b9. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:02.990083 19818 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:41:02.990374 19820 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 18:41:02.990540 19820 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 18:41:02.991194 19255 catalog_manager.cc:5495] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } health_report { overall_health: HEALTHY } } }
I20260929 18:41:03.009878 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.005112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:45487 (local address 127.0.0.1:58002)
0929 18:41:03.005349 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:03.005364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:03.005460 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:03.005827 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:03.005830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:03.005837 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:41:03.005952 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.005960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.007050 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.007053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.008053 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.008065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.008651 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.008655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.008746 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.008800 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.009550 (+ 750us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.009566 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 18:41:03.009637 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 18:41:03.009731 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260929 18:41:03.009877 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.005303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.11.118.195:45487)
0929 18:41:03.005414 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.005419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.005434 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:03.005611 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.005615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.005684 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.005799 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:03.006071 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.006933 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.008186 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.008538 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.008653 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.008947 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.009445 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.009554 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.009667 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.009734 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 18:41:03.013661 19557 raft_consensus.cc:1261] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Refusing update from remote peer 44b473ac8a0b4880bead2b3eb75760b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.013849 19418 raft_consensus.cc:1261] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Refusing update from remote peer 44b473ac8a0b4880bead2b3eb75760b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.014103 19778 consensus_queue.cc:1035] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.014710 19778 consensus_queue.cc:1035] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.015452 19780 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: TxnStatusTablet state changed. Reason: New leader 44b473ac8a0b4880bead2b3eb75760b9. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.015554 19780 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:03.016287 19779 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: TxnStatusTablet state changed. Reason: New leader 44b473ac8a0b4880bead2b3eb75760b9. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.016396 19779 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:03.018497 19818 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.018597 19818 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:41:03.019165 19779 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.019268 19779 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca6f02aacf8b4bbc84697deb7f723101: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:03.021685 19826 mvcc.cc:205] Tried to move back new op lower bound from 7334736949296734208 to 7334736949203693568. Current Snapshot: MvccSnapshot[applied={T|T < 7334736949296734208}]
I20260929 18:41:03.023468 19778 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.023556 19778 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P 44b473ac8a0b4880bead2b3eb75760b9: This TxnStatusTablet replica's current role is: LEADER
I20260929 18:41:03.025976 19780 tablet_replica.cc:439] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44b473ac8a0b4880bead2b3eb75760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } } }
I20260929 18:41:03.026109 19780 tablet_replica.cc:442] T 3c61766453144beb84dfa84549ef265d P ca663fed837e414db85dca97935e3a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 18:41:03.031910 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.027803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.11.118.194:40533)
0929 18:41:03.027932 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.027936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.028096 (+ 160us) server_negotiation.cc:407] Connection header received
0929 18:41:03.028234 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.028238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.028301 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.028413 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:03.028619 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.029404 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.030230 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.030601 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.030711 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.030946 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.031522 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.031638 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.031677 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.031769 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 18:41:03.043218 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.039784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:36703 (local address 127.11.118.195:45487)
0929 18:41:03.039950 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.039955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.040072 (+ 117us) server_negotiation.cc:407] Connection header received
0929 18:41:03.040236 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.040239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.040297 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.040383 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:03.040625 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.041133 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.042038 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.042319 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.042445 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.042741 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.042831 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.042915 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.043010 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.043066 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 18:41:03.043219 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.039663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:45487 (local address 127.11.118.194:36703)
0929 18:41:03.039979 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:03.040000 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:03.040130 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:03.040393 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:03.040396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:03.040404 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:41:03.040517 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.040522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.041245 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.041252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.041904 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.041913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.042447 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.042451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.042544 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.042597 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.042919 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.042928 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 18:41:03.042990 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 18:41:03.043078 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
I20260929 18:41:03.050323 19683 raft_consensus.cc:1261] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.050518 19418 raft_consensus.cc:1261] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.050815 19780 consensus_queue.cc:1035] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.051002 19806 consensus_queue.cc:1035] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.054188 19824 mvcc.cc:205] Tried to move back new op lower bound from 7334736949449793536 to 7334736948513431552. Current Snapshot: MvccSnapshot[applied={T|T < 7334736949449793536}]
I20260929 18:41:03.075730 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.071430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:38879 (local address 127.11.118.194:40533)
0929 18:41:03.071563 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.071568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.071807 (+ 239us) server_negotiation.cc:407] Connection header received
0929 18:41:03.072026 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.072029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.072084 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.072156 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:03.072459 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.073194 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.074493 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.074794 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.074890 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.075247 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.075345 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.075440 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.075527 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.075580 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 18:41:03.075789 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.071357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:40533 (local address 127.11.118.195:38879)
0929 18:41:03.071721 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:03.071737 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:03.071928 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:03.072172 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:03.072176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:03.072185 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 18:41:03.072317 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.072323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.073310 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.073313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.074359 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:03.074371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.074925 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.074928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:03.075025 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.075081 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.075443 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.075452 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 18:41:03.075519 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 18:41:03.075599 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
I20260929 18:41:03.103861 19261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.104056 19261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.106698 19261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Sending DeleteTablet for 3 replicas of tablet b01aa15d686946bbbee80b098e419097
I20260929 18:41:03.107502 19398 tablet_service.cc:1514] Processing DeleteTablet for tablet b01aa15d686946bbbee80b098e419097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49004
I20260929 18:41:03.107645 19538 tablet_service.cc:1514] Processing DeleteTablet for tablet b01aa15d686946bbbee80b098e419097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34152
I20260929 18:41:03.107898 19844 tablet_replica.cc:330] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: stopping tablet replica
I20260929 18:41:03.108028 19844 raft_consensus.cc:2229] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.108114 19678 tablet_service.cc:1514] Processing DeleteTablet for tablet b01aa15d686946bbbee80b098e419097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57988
I20260929 18:41:03.108156 19844 raft_consensus.cc:2258] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.108443 19846 tablet_replica.cc:330] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: stopping tablet replica
I20260929 18:41:03.108570 19846 raft_consensus.cc:2229] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.108683 19846 raft_consensus.cc:2258] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.109442 19846 ts_tablet_manager.cc:1905] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.112756 19846 ts_tablet_manager.cc:1918] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.112876 19846 log.cc:1196] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/b01aa15d686946bbbee80b098e419097
I20260929 18:41:03.113138 19846 ts_tablet_manager.cc:1939] T b01aa15d686946bbbee80b098e419097 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting consensus metadata
I20260929 18:41:03.113605 19844 ts_tablet_manager.cc:1905] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.113938 19248 catalog_manager.cc:4841] TS 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487): tablet b01aa15d686946bbbee80b098e419097 (table test-table [id=6d3f18c0a76e4058a279352bae871ce7]) successfully deleted
I20260929 18:41:03.115876 19845 tablet_replica.cc:330] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: stopping tablet replica
I20260929 18:41:03.116092 19845 raft_consensus.cc:2229] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:41:03.116508 19845 raft_consensus.cc:2258] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.117316 19845 ts_tablet_manager.cc:1905] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.117321 19844 ts_tablet_manager.cc:1918] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.117398 19844 log.cc:1196] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/b01aa15d686946bbbee80b098e419097
I20260929 18:41:03.117669 19844 ts_tablet_manager.cc:1939] T b01aa15d686946bbbee80b098e419097 P ca6f02aacf8b4bbc84697deb7f723101: Deleting consensus metadata
I20260929 18:41:03.120411 19845 ts_tablet_manager.cc:1918] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.121131 19845 log.cc:1196] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/b01aa15d686946bbbee80b098e419097
I20260929 18:41:03.121265 19248 catalog_manager.cc:4841] TS ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155): tablet b01aa15d686946bbbee80b098e419097 (table test-table [id=6d3f18c0a76e4058a279352bae871ce7]) successfully deleted
I20260929 18:41:03.121541 19845 ts_tablet_manager.cc:1939] T b01aa15d686946bbbee80b098e419097 P ca663fed837e414db85dca97935e3a13: Deleting consensus metadata
I20260929 18:41:03.123829 19248 catalog_manager.cc:4841] TS ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533): tablet b01aa15d686946bbbee80b098e419097 (table test-table [id=6d3f18c0a76e4058a279352bae871ce7]) successfully deleted
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 18:41:03.148445 19814 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:03.140185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.11.118.254:37517)
0929 18:41:03.140324 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.140329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.140631 (+ 302us) server_negotiation.cc:407] Connection header received
0929 18:41:03.140784 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.140787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.140844 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.140935 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:03.141844 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.142783 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.144106 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.144491 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.144600 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.148307 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57568: BlockingRecv error: recv got EOF from 127.0.0.1:57568 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM
I20260929 18:41:03.160982 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.150627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.11.118.254:37517)
0929 18:41:03.150733 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.150737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.150921 (+ 184us) server_negotiation.cc:407] Connection header received
0929 18:41:03.151020 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.151023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.151071 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.151155 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:03.151922 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.152721 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.153828 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.154174 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.154300 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.157302 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:03.157306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:03.157308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:03.157338 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:03.159071 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:03.159563 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:03.159566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:03.159569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:03.159655 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:03.159938 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:03.159941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:03.159943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:03.160154 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:03.160279 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.160531 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.160656 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 18:41:03.163923 19261 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:41:03.164412 19261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:03.169655 19398 tablet_service.cc:1467] Processing CreateTablet for tablet 38c7c726eb654d0a97994568967bc91a (DEFAULT_TABLE table=test-table [id=824d95c5339b4881a69986d048d5fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.169987 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c7c726eb654d0a97994568967bc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.172664 19773 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Bootstrap starting.
I20260929 18:41:03.172649 19678 tablet_service.cc:1467] Processing CreateTablet for tablet 38c7c726eb654d0a97994568967bc91a (DEFAULT_TABLE table=test-table [id=824d95c5339b4881a69986d048d5fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.172915 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c7c726eb654d0a97994568967bc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.173620 19773 tablet_bootstrap.cc:654] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.174861 19773 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: No bootstrap required, opened a new log
I20260929 18:41:03.174942 19773 ts_tablet_manager.cc:1397] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:03.175343 19773 raft_consensus.cc:348] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.175446 19773 raft_consensus.cc:374] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.175474 19773 raft_consensus.cc:729] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.175602 19773 consensus_queue.cc:260] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.175626 19538 tablet_service.cc:1467] Processing CreateTablet for tablet 38c7c726eb654d0a97994568967bc91a (DEFAULT_TABLE table=test-table [id=824d95c5339b4881a69986d048d5fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.175843 19773 ts_tablet_manager.cc:1428] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:41:03.175931 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c7c726eb654d0a97994568967bc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.178251 19772 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Bootstrap starting.
I20260929 18:41:03.178323 19774 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Bootstrap starting.
I20260929 18:41:03.179154 19774 tablet_bootstrap.cc:654] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.179232 19772 tablet_bootstrap.cc:654] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.180455 19772 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: No bootstrap required, opened a new log
I20260929 18:41:03.180547 19772 ts_tablet_manager.cc:1397] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 18:41:03.180963 19772 raft_consensus.cc:348] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.181067 19772 raft_consensus.cc:374] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.181097 19772 raft_consensus.cc:729] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca663fed837e414db85dca97935e3a13, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.181212 19772 consensus_queue.cc:260] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.181440 19772 ts_tablet_manager.cc:1428] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 18:41:03.184559 19774 tablet_bootstrap.cc:492] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: No bootstrap required, opened a new log
I20260929 18:41:03.184664 19774 ts_tablet_manager.cc:1397] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260929 18:41:03.185104 19774 raft_consensus.cc:348] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.185228 19774 raft_consensus.cc:374] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.185259 19774 raft_consensus.cc:729] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.185389 19774 consensus_queue.cc:260] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.185642 19774 ts_tablet_manager.cc:1428] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:03.511334 19840 raft_consensus.cc:482] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:03.511484 19840 raft_consensus.cc:504] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.511924 19840 leader_election.cc:288] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:03.512507 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c7c726eb654d0a97994568967bc91a" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101" is_pre_election: true
I20260929 18:41:03.512538 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c7c726eb654d0a97994568967bc91a" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9" is_pre_election: true
I20260929 18:41:03.512696 19418 raft_consensus.cc:2454] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:03.512799 19683 raft_consensus.cc:2454] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:03.513018 19492 leader_election.cc:302] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:03.513278 19840 raft_consensus.cc:2781] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:03.513360 19840 raft_consensus.cc:482] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:03.513411 19840 raft_consensus.cc:3037] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.514528 19840 raft_consensus.cc:504] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.514858 19840 leader_election.cc:288] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Requested vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:03.515362 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c7c726eb654d0a97994568967bc91a" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101"
I20260929 18:41:03.515547 19418 raft_consensus.cc:3037] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.515391 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c7c726eb654d0a97994568967bc91a" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9"
I20260929 18:41:03.515964 19683 raft_consensus.cc:3037] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.516764 19418 raft_consensus.cc:2454] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:03.517086 19492 leader_election.cc:302] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:03.517289 19840 raft_consensus.cc:2781] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:03.517313 19683 raft_consensus.cc:2454] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:03.517383 19840 raft_consensus.cc:686] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Becoming Leader. State: Replica: ca663fed837e414db85dca97935e3a13, State: Running, Role: LEADER
I20260929 18:41:03.517539 19840 consensus_queue.cc:237] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.519311 19256 catalog_manager.cc:5495] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca663fed837e414db85dca97935e3a13 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "ca663fed837e414db85dca97935e3a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } health_report { overall_health: UNKNOWN } } }
I20260929 18:41:03.552537 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:03.549276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34164 (local address 127.11.118.194:40533)
0929 18:41:03.549457 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:03.549462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:03.549477 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:03.549557 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:03.549560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:03.549612 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:03.549694 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:03.549944 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.550525 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.551392 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:03.551615 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:03.551722 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:03.551998 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:03.552075 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:03.552180 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:03.552319 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 18:41:03.552375 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:41:03.554665 19683 raft_consensus.cc:1261] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.554665 19418 raft_consensus.cc:1261] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.555195 19780 consensus_queue.cc:1035] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.555356 19840 consensus_queue.cc:1035] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.563426 19256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.563570 19256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.565929 19256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Sending DeleteTablet for 3 replicas of tablet 38c7c726eb654d0a97994568967bc91a
I20260929 18:41:03.566571 19398 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c7c726eb654d0a97994568967bc91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49004
I20260929 18:41:03.566659 19538 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c7c726eb654d0a97994568967bc91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34152
I20260929 18:41:03.566811 19844 tablet_replica.cc:330] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: stopping tablet replica
I20260929 18:41:03.566830 19678 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c7c726eb654d0a97994568967bc91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57988
I20260929 18:41:03.566883 19845 tablet_replica.cc:330] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: stopping tablet replica
I20260929 18:41:03.566959 19844 raft_consensus.cc:2229] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.566999 19846 tablet_replica.cc:330] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: stopping tablet replica
I20260929 18:41:03.567001 19845 raft_consensus.cc:2229] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:41:03.567111 19844 raft_consensus.cc:2258] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.567108 19846 raft_consensus.cc:2229] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.567236 19846 raft_consensus.cc:2258] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.567255 19845 raft_consensus.cc:2258] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.567978 19846 ts_tablet_manager.cc:1905] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.567989 19845 ts_tablet_manager.cc:1905] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.568017 19844 ts_tablet_manager.cc:1905] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.570008 19846 ts_tablet_manager.cc:1918] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:03.570119 19846 log.cc:1196] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/38c7c726eb654d0a97994568967bc91a
I20260929 18:41:03.570427 19846 ts_tablet_manager.cc:1939] T 38c7c726eb654d0a97994568967bc91a P 44b473ac8a0b4880bead2b3eb75760b9: Deleting consensus metadata
I20260929 18:41:03.570444 19845 ts_tablet_manager.cc:1918] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:03.570444 19844 ts_tablet_manager.cc:1918] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:03.570518 19845 log.cc:1196] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/38c7c726eb654d0a97994568967bc91a
I20260929 18:41:03.570529 19844 log.cc:1196] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/38c7c726eb654d0a97994568967bc91a
I20260929 18:41:03.570787 19845 ts_tablet_manager.cc:1939] T 38c7c726eb654d0a97994568967bc91a P ca663fed837e414db85dca97935e3a13: Deleting consensus metadata
I20260929 18:41:03.570814 19844 ts_tablet_manager.cc:1939] T 38c7c726eb654d0a97994568967bc91a P ca6f02aacf8b4bbc84697deb7f723101: Deleting consensus metadata
I20260929 18:41:03.571187 19248 catalog_manager.cc:4841] TS 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487): tablet 38c7c726eb654d0a97994568967bc91a (table test-table [id=824d95c5339b4881a69986d048d5fc48]) successfully deleted
I20260929 18:41:03.571581 19248 catalog_manager.cc:4841] TS ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155): tablet 38c7c726eb654d0a97994568967bc91a (table test-table [id=824d95c5339b4881a69986d048d5fc48]) successfully deleted
I20260929 18:41:03.571755 19248 catalog_manager.cc:4841] TS ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533): tablet 38c7c726eb654d0a97994568967bc91a (table test-table [id=824d95c5339b4881a69986d048d5fc48]) successfully deleted
I20260929 18:41:03.572158 19256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:41:03.572693 19256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:03.578115 19538 tablet_service.cc:1467] Processing CreateTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 (DEFAULT_TABLE table=test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.578110 19398 tablet_service.cc:1467] Processing CreateTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 (DEFAULT_TABLE table=test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.578346 19678 tablet_service.cc:1467] Processing CreateTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 (DEFAULT_TABLE table=test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:03.578514 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 162496a139ea4c01ae6ce3439f5c6c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.578555 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 162496a139ea4c01ae6ce3439f5c6c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.578610 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 162496a139ea4c01ae6ce3439f5c6c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:03.581116 19772 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Bootstrap starting.
I20260929 18:41:03.581312 19773 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Bootstrap starting.
I20260929 18:41:03.581432 19774 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Bootstrap starting.
I20260929 18:41:03.582018 19772 tablet_bootstrap.cc:654] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.582193 19773 tablet_bootstrap.cc:654] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.582197 19774 tablet_bootstrap.cc:654] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:03.583184 19772 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: No bootstrap required, opened a new log
I20260929 18:41:03.583281 19772 ts_tablet_manager.cc:1397] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 18:41:03.583349 19773 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: No bootstrap required, opened a new log
I20260929 18:41:03.583427 19773 ts_tablet_manager.cc:1397] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:03.583725 19772 raft_consensus.cc:348] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.583843 19772 raft_consensus.cc:374] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.583899 19772 raft_consensus.cc:729] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca663fed837e414db85dca97935e3a13, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.583905 19773 raft_consensus.cc:348] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.584048 19773 raft_consensus.cc:374] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.584086 19773 raft_consensus.cc:729] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f02aacf8b4bbc84697deb7f723101, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.584035 19772 consensus_queue.cc:260] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.584213 19773 consensus_queue.cc:260] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.584342 19774 tablet_bootstrap.cc:492] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: No bootstrap required, opened a new log
I20260929 18:41:03.584431 19774 ts_tablet_manager.cc:1397] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:03.584545 19773 ts_tablet_manager.cc:1428] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:41:03.584885 19774 raft_consensus.cc:348] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.584973 19772 ts_tablet_manager.cc:1428] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 18:41:03.585016 19774 raft_consensus.cc:374] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:03.585058 19774 raft_consensus.cc:729] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b473ac8a0b4880bead2b3eb75760b9, State: Initialized, Role: FOLLOWER
I20260929 18:41:03.585237 19774 consensus_queue.cc:260] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.585538 19774 ts_tablet_manager.cc:1428] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:41:03.614270 19806 raft_consensus.cc:482] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:03.614485 19806 raft_consensus.cc:504] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.615031 19806 leader_election.cc:288] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:03.615432 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "162496a139ea4c01ae6ce3439f5c6c59" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101" is_pre_election: true
I20260929 18:41:03.615492 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "162496a139ea4c01ae6ce3439f5c6c59" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9" is_pre_election: true
I20260929 18:41:03.615602 19418 raft_consensus.cc:2454] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:03.615617 19683 raft_consensus.cc:2454] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 0.
I20260929 18:41:03.615883 19492 leader_election.cc:302] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:03.616132 19806 raft_consensus.cc:2781] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:03.616192 19806 raft_consensus.cc:482] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:03.616220 19806 raft_consensus.cc:3037] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.617432 19806 raft_consensus.cc:504] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.617799 19806 leader_election.cc:288] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Requested vote from peers ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155), 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487)
I20260929 18:41:03.618350 19418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "162496a139ea4c01ae6ce3439f5c6c59" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f02aacf8b4bbc84697deb7f723101"
I20260929 18:41:03.618350 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "162496a139ea4c01ae6ce3439f5c6c59" candidate_uuid: "ca663fed837e414db85dca97935e3a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b473ac8a0b4880bead2b3eb75760b9"
I20260929 18:41:03.618491 19418 raft_consensus.cc:3037] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.618491 19683 raft_consensus.cc:3037] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:03.619692 19683 raft_consensus.cc:2454] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:03.619731 19418 raft_consensus.cc:2454] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca663fed837e414db85dca97935e3a13 in term 1.
I20260929 18:41:03.620034 19492 leader_election.cc:302] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca663fed837e414db85dca97935e3a13, ca6f02aacf8b4bbc84697deb7f723101; no voters:
I20260929 18:41:03.620255 19806 raft_consensus.cc:2781] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:03.620349 19806 raft_consensus.cc:686] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Becoming Leader. State: Replica: ca663fed837e414db85dca97935e3a13, State: Running, Role: LEADER
I20260929 18:41:03.620484 19806 consensus_queue.cc:237] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } }
I20260929 18:41:03.622129 19261 catalog_manager.cc:5495] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca663fed837e414db85dca97935e3a13 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "ca663fed837e414db85dca97935e3a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca663fed837e414db85dca97935e3a13" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 40533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 } health_report { overall_health: UNKNOWN } } }
I20260929 18:41:03.650694 19683 raft_consensus.cc:1261] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.650738 19418 raft_consensus.cc:1261] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Refusing update from remote peer ca663fed837e414db85dca97935e3a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:03.651121 19806 consensus_queue.cc:1035] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6f02aacf8b4bbc84697deb7f723101" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.651288 19780 consensus_queue.cc:1035] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b473ac8a0b4880bead2b3eb75760b9" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 45487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:03.685395 19256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.685549 19256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:03.687806 19256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a67a6683e4f46f39d6fb36b5fcaba71: Sending DeleteTablet for 3 replicas of tablet 162496a139ea4c01ae6ce3439f5c6c59
I20260929 18:41:03.688428 19398 tablet_service.cc:1514] Processing DeleteTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49004
I20260929 18:41:03.688515 19538 tablet_service.cc:1514] Processing DeleteTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:34152
I20260929 18:41:03.688637 19844 tablet_replica.cc:330] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: stopping tablet replica
I20260929 18:41:03.688723 19845 tablet_replica.cc:330] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: stopping tablet replica
I20260929 18:41:03.688719 19678 tablet_service.cc:1514] Processing DeleteTablet for tablet 162496a139ea4c01ae6ce3439f5c6c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:03 UTC) from {username='oryx', principal='oryx/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57988
I20260929 18:41:03.688786 19844 raft_consensus.cc:2229] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.688833 19845 raft_consensus.cc:2229] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:41:03.688910 19846 tablet_replica.cc:330] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: stopping tablet replica
I20260929 18:41:03.688905 19844 raft_consensus.cc:2258] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.689034 19846 raft_consensus.cc:2229] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:03.689033 19845 raft_consensus.cc:2258] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.689165 19846 raft_consensus.cc:2258] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:03.689824 19844 ts_tablet_manager.cc:1905] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.689886 19846 ts_tablet_manager.cc:1905] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.689885 19845 ts_tablet_manager.cc:1905] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:03.692068 19844 ts_tablet_manager.cc:1918] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.692167 19844 log.cc:1196] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/162496a139ea4c01ae6ce3439f5c6c59
I20260929 18:41:03.692287 19845 ts_tablet_manager.cc:1918] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.692287 19846 ts_tablet_manager.cc:1918] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 18:41:03.692370 19846 log.cc:1196] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/162496a139ea4c01ae6ce3439f5c6c59
I20260929 18:41:03.692381 19845 log.cc:1196] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/162496a139ea4c01ae6ce3439f5c6c59
I20260929 18:41:03.692456 19844 ts_tablet_manager.cc:1939] T 162496a139ea4c01ae6ce3439f5c6c59 P ca6f02aacf8b4bbc84697deb7f723101: Deleting consensus metadata
I20260929 18:41:03.692663 19845 ts_tablet_manager.cc:1939] T 162496a139ea4c01ae6ce3439f5c6c59 P ca663fed837e414db85dca97935e3a13: Deleting consensus metadata
I20260929 18:41:03.692663 19846 ts_tablet_manager.cc:1939] T 162496a139ea4c01ae6ce3439f5c6c59 P 44b473ac8a0b4880bead2b3eb75760b9: Deleting consensus metadata
I20260929 18:41:03.693339 19248 catalog_manager.cc:4841] TS 44b473ac8a0b4880bead2b3eb75760b9 (127.11.118.195:45487): tablet 162496a139ea4c01ae6ce3439f5c6c59 (table test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]) successfully deleted
I20260929 18:41:03.693529 19248 catalog_manager.cc:4841] TS ca663fed837e414db85dca97935e3a13 (127.11.118.194:40533): tablet 162496a139ea4c01ae6ce3439f5c6c59 (table test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]) successfully deleted
I20260929 18:41:03.693647 19248 catalog_manager.cc:4841] TS ca6f02aacf8b4bbc84697deb7f723101 (127.11.118.193:45155): tablet 162496a139ea4c01ae6ce3439f5c6c59 (table test-table [id=a6615b3fe5604f7eb58f68d544ef0dc6]) successfully deleted
I20260929 18:41:03.694008 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 19333
I20260929 18:41:03.701316 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 19473
I20260929 18:41:03.707937 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 19613
I20260929 18:41:03.714406 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 19229
2026-09-29T18:41:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3592 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19878](info): set up 2 sockets
Sep 29 18:41:03 dist-test-slave-cgv7 krb5kdc[19878](info): commencing operation
krb5kdc: starting...
W20260929 18:41:04.759079 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:41:04 dist-test-slave-cgv7 krb5kdc[19878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:41:04.886317 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37749
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:04.996774 19891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:04.997085 19891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:04.997157 19891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:05.001317 19891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:05.001413 19891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:05.001448 19891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:05.001488 19891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:05.001524 19891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:05.006965 19891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:05.008077 19891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:05.009128 19891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:05.017890 19897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:05.018006 19896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:05.017872 19899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:05.018159 19891 server_base.cc:1034] running on GCE node
I20260929 18:41:05.018868 19891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:05.020068 19891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:05.021283 19891 hybrid_clock.cc:648] HybridClock initialized: now 1790707265021269 us; error 27 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T18:41:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1319 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:41:05 dist-test-slave-cgv7 krb5kdc[19904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:41:05 dist-test-slave-cgv7 krb5kdc[19904](info): set up 2 sockets
Sep 29 18:41:05 dist-test-slave-cgv7 krb5kdc[19904](info): commencing operation
krb5kdc: starting...
W20260929 18:41:06.095252 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:41:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:41:06.222846 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34623
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:06.335348 19917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:06.335721 19917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:06.335800 19917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:06.340106 19917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:06.340217 19917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:06.340256 19917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:06.340296 19917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:06.340337 19917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:06.346073 19917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:06.347338 19917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:06.348409 19917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:06.357342 19923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:06.357326 19922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:06.357326 19925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:06.357468 19917 server_base.cc:1034] running on GCE node
I20260929 18:41:06.358212 19917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:06.359445 19917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:06.360634 19917 hybrid_clock.cc:648] HybridClock initialized: now 1790707266360618 us; error 31 us; skew 500 ppm
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:06.363584 19917 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:41:06.364480 19917 webserver.cc:460] Webserver started at http://127.11.118.254:42769/ using document root <none> and password file <none>
I20260929 18:41:06.365048 19917 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:06.365160 19917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:06.365391 19917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:06.367563 19917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "5a52cc41e1764c798b9063a9acb64208"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "9b9e8e10aa7f88d5dc3956f258907f35"
server_key_iv: "c27e7b5464ec1556322af3b484f547c6"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.368173 19917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "5a52cc41e1764c798b9063a9acb64208"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "9b9e8e10aa7f88d5dc3956f258907f35"
server_key_iv: "c27e7b5464ec1556322af3b484f547c6"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.372488 19917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:41:06.375073 19932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.376418 19917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:41:06.376586 19917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "5a52cc41e1764c798b9063a9acb64208"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "9b9e8e10aa7f88d5dc3956f258907f35"
server_key_iv: "c27e7b5464ec1556322af3b484f547c6"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.376744 19917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:06.400490 19917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:06.404107 19917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:06.404340 19917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:06.412097 19917 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:34623
I20260929 18:41:06.412117 19984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:34623 every 8 connection(s)
I20260929 18:41:06.413148 19917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:06.416043 19985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:06.419822 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 19917
I20260929 18:41:06.419942 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:41:06.420257 11739 external_mini_cluster.cc:1428] Setting key b1b4a43a8055a2fff6137cd872ba551f
I20260929 18:41:06.422643 19985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Bootstrap starting.
I20260929 18:41:06.425596 19985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:06.426589 19985 log.cc:824] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:06.429054 19985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: No bootstrap required, opened a new log
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:06.433691 19985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:06.434046 19985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:06.434120 19985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a52cc41e1764c798b9063a9acb64208, State: Initialized, Role: FOLLOWER
I20260929 18:41:06.434684 19985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:06.434865 19985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:06.434938 19985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:06.435052 19985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:06.436033 19985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:06.436452 19985 leader_election.cc:302] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a52cc41e1764c798b9063a9acb64208; no voters:
I20260929 18:41:06.436841 19985 leader_election.cc:288] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:41:06.436920 19990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:06.437112 19990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 LEADER]: Becoming Leader. State: Replica: 5a52cc41e1764c798b9063a9acb64208, State: Running, Role: LEADER
I20260929 18:41:06.437268 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:06.421857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44338 (local address 127.11.118.254:34623)
0929 18:41:06.422375 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:06.422395 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:06.422440 (+ 45us) server_negotiation.cc:407] Connection header received
0929 18:41:06.423303 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:06.423329 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:06.423789 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:06.424207 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:06.425034 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.426645 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.427959 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.428652 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.428830 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:06.431534 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:06.431542 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:06.431563 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:06.431608 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:06.433809 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.434419 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.434425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.434431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.434582 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.434893 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.434898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.434900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.435442 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:06.435663 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:06.435970 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 18:41:06.436305 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":144,"threads_started":1}
I20260929 18:41:06.437407 19990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:06.438196 19985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:06.438601 19992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a52cc41e1764c798b9063a9acb64208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } } }
I20260929 18:41:06.438611 19990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a52cc41e1764c798b9063a9acb64208. Latest consensus state: current_term: 1 leader_uuid: "5a52cc41e1764c798b9063a9acb64208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } } }
I20260929 18:41:06.438789 19992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:06.438803 19990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:06.439507 19994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:06.442770 19994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:06.448449 19994 catalog_manager.cc:1347] Generated new cluster ID: eb51dff6f098424cbcb441d9f56905f0
I20260929 18:41:06.448535 19994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:06.458309 19994 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:41:06.459306 19994 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:06.466442 19994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Generated new TSK 0
I20260929 18:41:06.467144 19994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:41:06.497843 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:06.628839 20011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:06.629120 20011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:06.629164 20011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:06.633306 20011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:06.633378 20011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:06.633478 20011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:06.638970 20011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:06.640103 20011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:06.641228 20011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:06.651036 20019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:06.651094 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:06.651366 20017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:06.651547 20011 server_base.cc:1034] running on GCE node
I20260929 18:41:06.651988 20011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:06.652705 20011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:06.653929 20011 hybrid_clock.cc:648] HybridClock initialized: now 1790707266653904 us; error 40 us; skew 500 ppm
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:06.657027 20011 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:41:06.657949 20011 webserver.cc:460] Webserver started at http://127.11.118.193:39417/ using document root <none> and password file <none>
I20260929 18:41:06.658613 20011 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:06.658715 20011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:06.658962 20011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:06.661110 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "76dc03ded8b74404819a5079cbe2a488"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "b951481c4f797b4fc71a8c7d68e7e4cf"
server_key_iv: "e8ecca1385c2b8ba97e35a93cecbc278"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.661738 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "76dc03ded8b74404819a5079cbe2a488"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "b951481c4f797b4fc71a8c7d68e7e4cf"
server_key_iv: "e8ecca1385c2b8ba97e35a93cecbc278"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.665799 20011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260929 18:41:06.668555 20026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.669811 20011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 18:41:06.669957 20011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "76dc03ded8b74404819a5079cbe2a488"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "b951481c4f797b4fc71a8c7d68e7e4cf"
server_key_iv: "e8ecca1385c2b8ba97e35a93cecbc278"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.670082 20011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:06.703168 20011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:06.706456 20011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:06.706688 20011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:06.707299 20011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:06.708158 20011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:06.708230 20011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.708302 20011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:06.708351 20011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.717640 20011 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:44417
I20260929 18:41:06.717656 20139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:44417 every 8 connection(s)
I20260929 18:41:06.718674 20011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:06.725735 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20011
I20260929 18:41:06.725849 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:41:06.726169 11739 external_mini_cluster.cc:1428] Setting key 937b623665535165ed30a65742cdcee5
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:06.733953 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:06.720600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:35079 (local address 127.11.118.254:34623)
0929 18:41:06.720764 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:06.720768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:06.721652 (+ 884us) server_negotiation.cc:407] Connection header received
0929 18:41:06.723045 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:06.723048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:06.723106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:06.723217 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:06.724479 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.725024 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.726370 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.726665 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.726789 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:06.729905 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:06.729908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:06.729910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:06.729934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:06.731416 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.732110 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.732113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.732114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.732196 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.732651 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.732653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.732655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.732850 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:06.732941 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:06.733705 (+ 764us) server_negotiation.cc:299] Negotiation successful
0929 18:41:06.733824 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:41:06.735252 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:06.720989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:35079)
0929 18:41:06.721462 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:06.721508 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:06.722820 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:06.723404 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:06.723411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:06.723822 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:06.724331 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:06.724345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.725169 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.725177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:06.726196 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:06.726228 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.726847 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.726852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:06.727055 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:06.727798 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:06.727819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:06.729700 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:06.731551 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:06.731558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:06.731578 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:06.731991 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:06.732294 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:06.732298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:06.732301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:06.732525 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:06.732977 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:06.732986 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:06.733385 (+ 399us) client_negotiation.cc:769] Sending connection context
0929 18:41:06.733641 (+ 256us) client_negotiation.cc:240] Negotiation successful
0929 18:41:06.734055 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:41:06.736593 20140 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34623
I20260929 18:41:06.736850 20140 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:06.737318 20140 heartbeater.cc:511] Master 127.11.118.254:34623 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:41:06.739053 19949 ts_manager.cc:194] Registered new tserver with Master: 76dc03ded8b74404819a5079cbe2a488 (127.11.118.193:44417)
I20260929 18:41:06.740772 19949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:35079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:41:06.756680 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:06.867554 20145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:06.867867 20145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:06.867944 20145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:06.872013 20145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:06.872118 20145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:06.872247 20145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:06.877723 20145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:06.878819 20145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:06.879802 20145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:06.889609 20150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:06.889624 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:06.889707 20145 server_base.cc:1034] running on GCE node
W20260929 18:41:06.889686 20151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:06.890477 20145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:06.891089 20145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:06.892266 20145 hybrid_clock.cc:648] HybridClock initialized: now 1790707266892255 us; error 25 us; skew 500 ppm
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:06.895282 20145 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:41:06.896235 20145 webserver.cc:460] Webserver started at http://127.11.118.194:40745/ using document root <none> and password file <none>
I20260929 18:41:06.896761 20145 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:06.896821 20145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:06.896983 20145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:06.899106 20145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "25eca81b1225470087402a98a015931f"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "8816eb6af916d8d64624f61c316140a4"
server_key_iv: "8c1ef8b60a094e01806231e0220a4d3d"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.899689 20145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "25eca81b1225470087402a98a015931f"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "8816eb6af916d8d64624f61c316140a4"
server_key_iv: "8c1ef8b60a094e01806231e0220a4d3d"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.903453 20145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 18:41:06.906134 20160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.907496 20145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 18:41:06.907644 20145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "25eca81b1225470087402a98a015931f"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "8816eb6af916d8d64624f61c316140a4"
server_key_iv: "8c1ef8b60a094e01806231e0220a4d3d"
server_key_version: "encryptionkey@0"
I20260929 18:41:06.907766 20145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:06.929152 20145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:06.932668 20145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:06.932883 20145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:06.933562 20145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:06.934473 20145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:06.934523 20145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.934598 20145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:06.934633 20145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:06.943931 20145 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41541
I20260929 18:41:06.943962 20273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41541 every 8 connection(s)
I20260929 18:41:06.944823 20145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:06.953682 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20145
I20260929 18:41:06.953832 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:41:06.954100 11739 external_mini_cluster.cc:1428] Setting key a23cc140d33cf2fc6c0edc361b4b6a8e
Sep 29 18:41:06 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707266, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:06.959973 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:06.946657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38745 (local address 127.11.118.254:34623)
0929 18:41:06.946820 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:06.946823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:06.947651 (+ 828us) server_negotiation.cc:407] Connection header received
0929 18:41:06.948991 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:06.948995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:06.949056 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:06.949155 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:06.950364 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.950899 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.952308 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.952556 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.952698 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:06.955734 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:06.955740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:06.955747 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:06.955794 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:06.957516 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.958059 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.958065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.958070 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.958181 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:06.958642 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:06.958648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:06.958653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:06.958919 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:06.959037 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:06.959579 (+ 542us) server_negotiation.cc:299] Negotiation successful
0929 18:41:06.959731 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 18:41:06.960777 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:06.947025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:38745)
0929 18:41:06.947461 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:06.947502 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:06.948775 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:06.949332 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:06.949339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:06.949693 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:06.950173 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:06.950187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.951026 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.951030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:06.952160 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:06.952169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:06.952738 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:06.952743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:06.952914 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:06.953534 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:06.953551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:06.955579 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:06.957646 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:06.957654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:06.957672 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:06.957951 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:06.958365 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:06.958367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:06.958368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:06.958531 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:06.959047 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:06.959053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:06.959305 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 18:41:06.959466 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 18:41:06.959816 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260929 18:41:06.962108 20274 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34623
I20260929 18:41:06.962445 20274 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:06.962977 20274 heartbeater.cc:511] Master 127.11.118.254:34623 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:41:06.964107 19949 ts_manager.cc:194] Registered new tserver with Master: 25eca81b1225470087402a98a015931f (127.11.118.194:41541)
I20260929 18:41:06.964707 19949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:38745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:41:06.983435 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:07.096663 20279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:07.096997 20279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:07.097092 20279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:07.101418 20279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:07.101526 20279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:07.101619 20279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:07.107228 20279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:07.108433 20279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:07.109570 20279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:07.119922 20287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.119957 20284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.119949 20285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.120119 20279 server_base.cc:1034] running on GCE node
I20260929 18:41:07.120718 20279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:07.121464 20279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:07.122651 20279 hybrid_clock.cc:648] HybridClock initialized: now 1790707267122620 us; error 60 us; skew 500 ppm
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707267, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:07.125403 20279 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:41:07.126436 20279 webserver.cc:460] Webserver started at http://127.11.118.195:35613/ using document root <none> and password file <none>
I20260929 18:41:07.127027 20279 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:07.127125 20279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:07.127369 20279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:07.129575 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "fbc5c6f6f231495c9d5fbcf09677c8d2"
format_stamp: "Formatted at 2026-09-29 18:41:07 on dist-test-slave-cgv7"
server_key: "ad2fe58c1ce6d334689739639eae2eec"
server_key_iv: "47a5d76c6f5cced2e3edf49d421ce59e"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.130173 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "fbc5c6f6f231495c9d5fbcf09677c8d2"
format_stamp: "Formatted at 2026-09-29 18:41:07 on dist-test-slave-cgv7"
server_key: "ad2fe58c1ce6d334689739639eae2eec"
server_key_iv: "47a5d76c6f5cced2e3edf49d421ce59e"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.134155 20279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:41:07.136829 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.138384 20279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:41:07.138530 20279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "fbc5c6f6f231495c9d5fbcf09677c8d2"
format_stamp: "Formatted at 2026-09-29 18:41:07 on dist-test-slave-cgv7"
server_key: "ad2fe58c1ce6d334689739639eae2eec"
server_key_iv: "47a5d76c6f5cced2e3edf49d421ce59e"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.138654 20279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:07.158216 20279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:07.161439 20279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:07.161662 20279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:07.162334 20279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:07.163214 20279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:07.163288 20279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.163360 20279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:07.163404 20279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.173202 20279 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:45531
I20260929 18:41:07.173219 20407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:45531 every 8 connection(s)
I20260929 18:41:07.174249 20279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:07.180250 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20279
I20260929 18:41:07.180425 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:41:07.180835 11739 external_mini_cluster.cc:1428] Setting key 8705cfa636ccf91e42bd1349b48404c6
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707267, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:07.188867 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.176214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:38681 (local address 127.11.118.254:34623)
0929 18:41:07.176353 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.176365 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.177259 (+ 894us) server_negotiation.cc:407] Connection header received
0929 18:41:07.178693 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.178696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.178758 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.178838 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.180287 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.181060 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.182608 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.182856 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.182975 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.185398 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:07.185401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:07.185404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:07.185437 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:07.186832 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:07.187354 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:07.187358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:07.187360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:07.187440 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:07.187790 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:07.187795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:07.187796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:07.187973 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:07.188093 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:07.188588 (+ 495us) server_negotiation.cc:299] Negotiation successful
0929 18:41:07.188705 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:41:07.189695 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.176598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:38681)
0929 18:41:07.177096 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.177141 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.178456 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.179022 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.179030 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.179520 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.180117 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.180133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.181199 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.181208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.182455 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.182465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.183014 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.183018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.183206 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.183786 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.183803 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.185228 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:07.186958 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:07.186963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:07.186978 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:07.187247 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:07.187540 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:07.187543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:07.187552 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:07.187692 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:07.188093 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:07.188101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:07.188351 (+ 250us) client_negotiation.cc:769] Sending connection context
0929 18:41:07.188538 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 18:41:07.188881 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":121,"threads_started":1}
I20260929 18:41:07.190851 20408 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34623
I20260929 18:41:07.191094 20408 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:07.191586 20408 heartbeater.cc:511] Master 127.11.118.254:34623 requested a full tablet report, sending...
I20260929 18:41:07.192598 19949 ts_manager.cc:194] Registered new tserver with Master: fbc5c6f6f231495c9d5fbcf09677c8d2 (127.11.118.195:45531)
I20260929 18:41:07.193266 19949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:38681
I20260929 18:41:07.194828 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
Entry for principal oryx/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab.
WARNING: no policy specified for oryx/127.11.118.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
Entry for principal oryx/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab.
WARNING: no policy specified for oryx/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.118.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
Entry for principal oryx/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab.
I20260929 18:41:07.268833 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20011
I20260929 18:41:07.274453 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20145
I20260929 18:41:07.279887 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20279
I20260929 18:41:07.285382 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 19917
I20260929 18:41:07.291597 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34623
--webserver_interface=127.11.118.254
--webserver_port=42769
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:07.405146 20417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:07.405450 20417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:07.405493 20417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:07.409603 20417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:07.409669 20417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:07.409689 20417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:07.409706 20417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:07.409722 20417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:07.415608 20417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=42769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:07.416711 20417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:07.417711 20417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:07.426512 20423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.426765 20417 server_base.cc:1034] running on GCE node
W20260929 18:41:07.426532 20422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.426512 20425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.427425 20417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:07.428490 20417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:07.429663 20417 hybrid_clock.cc:648] HybridClock initialized: now 1790707267429645 us; error 29 us; skew 500 ppm
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707267, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:07.432392 20417 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:41:07.433418 20417 webserver.cc:460] Webserver started at http://127.11.118.254:42769/ using document root <none> and password file <none>
I20260929 18:41:07.433992 20417 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:07.434108 20417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:07.438254 20417 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:41:07.440626 20432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.441957 20417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 18:41:07.442098 20417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "5a52cc41e1764c798b9063a9acb64208"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "9b9e8e10aa7f88d5dc3956f258907f35"
server_key_iv: "c27e7b5464ec1556322af3b484f547c6"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.442764 20417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:07.460043 20417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:07.463617 20417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:07.463840 20417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:07.471498 20417 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:34623
I20260929 18:41:07.471607 20484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:34623 every 8 connection(s)
I20260929 18:41:07.472821 20417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:07.477459 20485 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:41:07.478449 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20417
I20260929 18:41:07.478979 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:44417
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:34623
--webserver_port=39417
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:41:07.479658 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Bootstrap starting.
I20260929 18:41:07.496990 20485 log.cc:824] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:07.503880 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:41:07.504288 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Bootstrap complete.
I20260929 18:41:07.508574 20485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:07.509232 20485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a52cc41e1764c798b9063a9acb64208, State: Initialized, Role: FOLLOWER
I20260929 18:41:07.509963 20485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:07.510120 20485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:07.510218 20485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:07.510340 20485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:41:07.512976 20485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:07.513375 20485 leader_election.cc:302] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a52cc41e1764c798b9063a9acb64208; no voters:
I20260929 18:41:07.513741 20485 leader_election.cc:288] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 18:41:07.513833 20489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 18:41:07.514041 20489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [term 2 LEADER]: Becoming Leader. State: Replica: 5a52cc41e1764c798b9063a9acb64208, State: Running, Role: LEADER
I20260929 18:41:07.514433 20489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } }
I20260929 18:41:07.515044 20485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:07.515410 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a52cc41e1764c798b9063a9acb64208. Latest consensus state: current_term: 2 leader_uuid: "5a52cc41e1764c798b9063a9acb64208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } } }
I20260929 18:41:07.515606 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:07.515805 20490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5a52cc41e1764c798b9063a9acb64208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52cc41e1764c798b9063a9acb64208" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34623 } } }
I20260929 18:41:07.515982 20490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:07.516062 20493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:07.519380 20493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:07.520149 20493 catalog_manager.cc:1259] Loaded cluster ID: eb51dff6f098424cbcb441d9f56905f0
I20260929 18:41:07.520213 20493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:07.522620 20493 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:07.523289 20493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5a52cc41e1764c798b9063a9acb64208: Loaded TSK: 0
I20260929 18:41:07.524612 20493 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:41:07.597654 20487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:07.598014 20487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:07.598095 20487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:07.602492 20487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:07.602607 20487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:07.602715 20487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:07.608714 20487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.193.keytab
--principal=oryx/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:44417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=39417
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:07.609843 20487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:07.610926 20487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:07.621466 20512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.621459 20514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.621639 20487 server_base.cc:1034] running on GCE node
W20260929 18:41:07.621459 20511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.622442 20487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:07.623225 20487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:07.624399 20487 hybrid_clock.cc:648] HybridClock initialized: now 1790707267624372 us; error 56 us; skew 500 ppm
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707267, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:07.627585 20487 init.cc:377] Logged in from keytab as oryx/127.11.118.193@KRBTEST.COM (short username oryx)
I20260929 18:41:07.628677 20487 webserver.cc:460] Webserver started at http://127.11.118.193:39417/ using document root <none> and password file <none>
I20260929 18:41:07.629261 20487 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:07.629336 20487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:07.633484 20487 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:07.635810 20521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.637180 20487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:41:07.637280 20487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "76dc03ded8b74404819a5079cbe2a488"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "b951481c4f797b4fc71a8c7d68e7e4cf"
server_key_iv: "e8ecca1385c2b8ba97e35a93cecbc278"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.637852 20487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:07.655547 20487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:07.658975 20487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:07.659184 20487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:07.659842 20487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:07.660733 20487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:07.660782 20487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.660851 20487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:07.660883 20487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.671432 20487 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:44417
I20260929 18:41:07.671676 20634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:44417 every 8 connection(s)
I20260929 18:41:07.672840 20487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:07.676406 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20487
I20260929 18:41:07.677054 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:41541
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:34623
--webserver_port=40745
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:07.691905 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.676507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:51283)
0929 18:41:07.678652 (+ 2145us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.678697 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.680301 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.681689 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.681702 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.682306 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.683059 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.683084 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.684552 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.684558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.686170 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.686186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.687088 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.687097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.687345 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.688026 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.688055 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.690871 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1891,"thread_start_us":174,"threads_started":1}
W20260929 18:41:07.691900 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.676301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:51283 (local address 127.11.118.254:34623)
0929 18:41:07.677904 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.677920 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.678830 (+ 910us) server_negotiation.cc:407] Connection header received
0929 18:41:07.680789 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.680801 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.681179 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.681514 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.683278 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.684398 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.686363 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.686876 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.687027 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.691091 (+ 4064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:51283: BlockingRecv error: recv got EOF from 127.11.118.193:51283 (error 108)
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":139,"threads_started":1}
W20260929 18:41:07.692373 20635 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:34623 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.118.254:34623: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:07.698666 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.692933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:51311)
0929 18:41:07.693088 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.693103 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.693191 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.693429 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.693432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.693597 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.693714 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.693719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.694537 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.694543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.695548 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.695559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.696004 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.696009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.696104 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.696551 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.696564 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.698520 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260929 18:41:07.698932 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.693038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:51311 (local address 127.11.118.254:34623)
0929 18:41:07.693152 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.693155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.693167 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:41:07.693285 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.693287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.693326 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.693394 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.693846 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.694425 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.695680 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.695898 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.695979 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.698660 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:51311: BlockingRecv error: recv got EOF from 127.11.118.193:51311 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:07.704552 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.699199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:59621)
0929 18:41:07.699323 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.699338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.699413 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.699684 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.699687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.699860 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.699951 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.699957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.700670 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.700673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.701484 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.701495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.701909 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.701912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.701987 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.702496 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.702512 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.704399 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 18:41:07.704670 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.699294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:59621 (local address 127.11.118.254:34623)
0929 18:41:07.699447 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.699451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.699465 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:07.699505 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.699510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.699558 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.699645 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.700079 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.700559 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.701597 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.701806 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.701884 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.704520 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:59621: BlockingRecv error: recv got EOF from 127.11.118.193:59621 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":116,"threads_started":1}
W20260929 18:41:07.704931 20635 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:41:07.795876 20639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:07.796159 20639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:07.796204 20639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:07.800350 20639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:07.800419 20639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:07.800519 20639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:07.805811 20639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.194.keytab
--principal=oryx/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:41541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=40745
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:07.806876 20639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:07.807881 20639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:07.817845 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.817845 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:07.817845 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:07.818050 20639 server_base.cc:1034] running on GCE node
I20260929 18:41:07.818501 20639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:07.819141 20639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:07.820297 20639 hybrid_clock.cc:648] HybridClock initialized: now 1790707267820268 us; error 46 us; skew 500 ppm
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707267, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:07.823024 20639 init.cc:377] Logged in from keytab as oryx/127.11.118.194@KRBTEST.COM (short username oryx)
I20260929 18:41:07.823993 20639 webserver.cc:460] Webserver started at http://127.11.118.194:40745/ using document root <none> and password file <none>
I20260929 18:41:07.824548 20639 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:07.824651 20639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:07.828382 20639 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260929 18:41:07.830456 20655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.831670 20639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260929 18:41:07.831790 20639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "25eca81b1225470087402a98a015931f"
format_stamp: "Formatted at 2026-09-29 18:41:06 on dist-test-slave-cgv7"
server_key: "8816eb6af916d8d64624f61c316140a4"
server_key_iv: "8c1ef8b60a094e01806231e0220a4d3d"
server_key_version: "encryptionkey@0"
I20260929 18:41:07.832300 20639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:07.846813 20639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:07.849918 20639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:07.850128 20639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:07.850775 20639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:07.851614 20639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:07.851662 20639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.851701 20639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:07.851716 20639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:07.861137 20639 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41541
I20260929 18:41:07.861162 20768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41541 every 8 connection(s)
I20260929 18:41:07.862167 20639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:07.864491 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20639
I20260929 18:41:07.864912 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:45531
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:34623
--webserver_port=35613
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 18:41:07.874109 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.864124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:34143 (local address 127.11.118.254:34623)
0929 18:41:07.864306 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.864311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.866171 (+ 1860us) server_negotiation.cc:407] Connection header received
0929 18:41:07.867089 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.867093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.867165 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.867270 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.868483 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.869013 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.870196 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.870472 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.870585 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.873983 (+ 3398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:34143: BlockingRecv error: recv got EOF from 127.11.118.194:34143 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:41:07.874848 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.864573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:34143)
0929 18:41:07.865168 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.865252 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.866860 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.867464 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.867471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.867849 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.868328 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.868341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.869144 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.869149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.870040 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.870050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.870626 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.870630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.870812 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.871429 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.871446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.873858 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":142,"threads_started":1}
W20260929 18:41:07.875358 20769 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:34623 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.118.254:34623: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:07.881599 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.875849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:48077)
0929 18:41:07.876011 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.876025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.876113 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.876356 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.876358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.876528 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.876645 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.876651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.877382 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.877385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.878338 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.878348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.878848 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.878852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.878950 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.879511 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.879523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.881445 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260929 18:41:07.881686 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.875938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:48077 (local address 127.11.118.254:34623)
0929 18:41:07.876051 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.876055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.876068 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:07.876204 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.876206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.876245 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.876314 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.876774 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.877280 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.878457 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.878717 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.878847 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.881569 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:48077: BlockingRecv error: recv got EOF from 127.11.118.194:48077 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:07 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:07.887717 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:07.882335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:47567)
0929 18:41:07.882485 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:07.882498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:07.882564 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:07.882786 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:07.882789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:07.882950 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:07.883035 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.883041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.883765 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.883769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.884633 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:07.884643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.885093 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.885096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:07.885185 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.885644 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:07.885659 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:07.887582 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260929 18:41:07.887928 20769 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:41:07.887827 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:07.882435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:47567 (local address 127.11.118.254:34623)
0929 18:41:07.882543 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:07.882546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:07.882558 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:41:07.882647 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:07.882649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:07.882685 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:07.882751 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:07.883155 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.883625 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.884751 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:07.884976 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:07.885058 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:07.887704 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:47567: BlockingRecv error: recv got EOF from 127.11.118.194:47567 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 18:41:07.982589 20772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:07.982852 20772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:07.982896 20772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:07.987004 20772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:07.987071 20772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:07.987169 20772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:07.992691 20772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/krb5kdc/oryx_127.11.118.195.keytab
--principal=oryx/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:45531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=35613
--tserver_master_addrs=127.11.118.254:34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:07.993814 20772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:07.994937 20772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:41:08.005095 20772 server_base.cc:1034] running on GCE node
W20260929 18:41:08.005054 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:08.005034 20780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:08.005039 20778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:08.005774 20772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:08.006503 20772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:08.007690 20772 hybrid_clock.cc:648] HybridClock initialized: now 1790707268007669 us; error 30 us; skew 500 ppm
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707268, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:08.010676 20772 init.cc:377] Logged in from keytab as oryx/127.11.118.195@KRBTEST.COM (short username oryx)
I20260929 18:41:08.011646 20772 webserver.cc:460] Webserver started at http://127.11.118.195:35613/ using document root <none> and password file <none>
I20260929 18:41:08.012215 20772 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:08.012313 20772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:08.016391 20772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:41:08.018694 20787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:08.020066 20772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:41:08.020198 20772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "fbc5c6f6f231495c9d5fbcf09677c8d2"
format_stamp: "Formatted at 2026-09-29 18:41:07 on dist-test-slave-cgv7"
server_key: "ad2fe58c1ce6d334689739639eae2eec"
server_key_iv: "47a5d76c6f5cced2e3edf49d421ce59e"
server_key_version: "encryptionkey@0"
I20260929 18:41:08.020745 20772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:08.047575 20772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:08.050828 20772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:08.051049 20772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:08.051684 20772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:08.052537 20772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:08.052613 20772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:08.052680 20772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:08.052733 20772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:08.062024 20772 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:45531
I20260929 18:41:08.062091 20900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:45531 every 8 connection(s)
I20260929 18:41:08.063143 20772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:08.072290 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20772
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 18:41:08.074867 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:08.064992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:46493 (local address 127.11.118.254:34623)
0929 18:41:08.065145 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.065148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.065917 (+ 769us) server_negotiation.cc:407] Connection header received
0929 18:41:08.067313 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.067316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.067376 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.067485 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.068669 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.069195 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.070422 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.070654 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.070767 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.074762 (+ 3995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:46493: BlockingRecv error: recv got EOF from 127.11.118.195:46493 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:41:08.075609 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.065321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:46493)
0929 18:41:08.065758 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:08.065801 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:08.067076 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:08.067665 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:08.067673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:08.068023 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:08.068512 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.068526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.069324 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.069332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.070218 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.070227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.070827 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.070831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.071008 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.071613 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:08.071629 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:08.074639 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
W20260929 18:41:08.076205 20901 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:34623 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.118.254:34623: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:08.082124 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.076655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:60447)
0929 18:41:08.076774 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:08.076789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:08.076865 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:08.077108 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:08.077111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:08.077278 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:08.077408 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.077415 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.078447 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.078450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.079045 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.079055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.079474 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.079478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.079557 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.079916 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:08.079935 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:08.081960 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 18:41:08.082887 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:08.076743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:60447 (local address 127.11.118.254:34623)
0929 18:41:08.076870 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.076875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.076891 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:41:08.076979 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.076981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.077017 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.077072 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.077579 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.078333 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.079162 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.079361 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.079469 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.082126 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:60447: BlockingRecv error: recv got EOF from 127.11.118.195:60447 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:41:08.083786 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.073402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.11.118.254:34623)
0929 18:41:08.073535 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.073549 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.073891 (+ 342us) server_negotiation.cc:407] Connection header received
0929 18:41:08.074050 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.074052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.074092 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.074171 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.075098 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.075835 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.076938 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.077287 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.077405 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.079049 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:08.079056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:08.079081 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:08.079121 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:08.081099 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:08.081545 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:08.081552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:08.081559 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:08.081765 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:08.082103 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:08.082106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:08.082108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:08.082618 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:08.082836 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:08.083358 (+ 522us) server_negotiation.cc:299] Negotiation successful
0929 18:41:08.083622 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:08.089130 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.082722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:47115)
0929 18:41:08.083217 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:08.083232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:08.083337 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:08.083650 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:08.083653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:08.083810 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:08.083940 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.083949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.085530 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.085532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.086214 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.086224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.086789 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.086793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.086864 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.087363 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:08.087374 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:08.089030 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415}
W20260929 18:41:08.089279 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:08.082813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:47115 (local address 127.11.118.254:34623)
0929 18:41:08.083117 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.083120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.083280 (+ 160us) server_negotiation.cc:407] Connection header received
0929 18:41:08.083435 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.083438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.083475 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.083521 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.084249 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.084723 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.086404 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.086646 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.086749 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.089156 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:47115: BlockingRecv error: recv got EOF from 127.11.118.195:47115 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":78,"threads_started":1}
W20260929 18:41:08.089375 20901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:08.711189 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.705614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:41975)
0929 18:41:08.705892 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:08.705905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:08.705994 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:08.706228 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:08.706230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:08.706463 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:08.706589 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.706595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.707317 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.707320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.707909 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.707917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.708338 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.708340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.708413 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.708996 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:08.709008 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:08.711077 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
W20260929 18:41:08.711387 20907 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:08.705714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:41975 (local address 127.11.118.254:34623)
0929 18:41:08.705969 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.705972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.705986 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:08.706081 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.706083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.706138 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.706218 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.706693 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.707223 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.708015 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.708240 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.708317 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.711215 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:41975: BlockingRecv error: recv got EOF from 127.11.118.193:41975 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":104,"threads_started":1}
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:08 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:08.894553 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:08.888699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:44885)
0929 18:41:08.889000 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:08.889015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:08.889115 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:08.889366 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:08.889368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:08.889540 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:08.889633 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.889666 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.890639 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.890643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.891227 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:08.891236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.891702 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.891707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:08.891792 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.892288 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:08.892299 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:08.894371 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":113,"threads_started":1}
W20260929 18:41:08.894620 20907 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:08.888793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:44885 (local address 127.11.118.254:34623)
0929 18:41:08.888951 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:08.888955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:08.889110 (+ 155us) server_negotiation.cc:407] Connection header received
0929 18:41:08.889187 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:08.889190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:08.889247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:08.889332 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:08.889788 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.890515 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.891355 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:08.891584 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:08.891700 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:08.894507 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:44885: BlockingRecv error: recv got EOF from 127.11.118.194:44885 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:09.096244 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:09.090127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:37799)
0929 18:41:09.090446 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:09.090459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:09.090629 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:09.090901 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:09.090904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:09.091120 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:09.091211 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.091216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.091960 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.091967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.092691 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.092702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.093335 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.093338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.093443 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.094003 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:09.094020 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:09.096118 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
W20260929 18:41:09.096407 20907 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:09.090236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:37799 (local address 127.11.118.254:34623)
0929 18:41:09.090392 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:09.090397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:09.090565 (+ 168us) server_negotiation.cc:407] Connection header received
0929 18:41:09.090730 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:09.090734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:09.090786 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:09.090877 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:09.091328 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.091853 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.092844 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.093194 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.093319 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.096276 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:37799: BlockingRecv error: recv got EOF from 127.11.118.195:37799 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:09.717538 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:09.711921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:58705)
0929 18:41:09.712210 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:09.712223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:09.712311 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:09.712568 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:09.712571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:09.712747 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:09.712842 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.712846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.713555 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.713558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.714106 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.714114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.714687 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.714690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.714771 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.715304 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:09.715316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:09.717368 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
W20260929 18:41:09.717705 20911 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:09.712056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:58705 (local address 127.11.118.254:34623)
0929 18:41:09.712284 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:09.712288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:09.712301 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:09.712400 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:09.712402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:09.712455 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:09.712530 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:09.712942 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.713465 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.714207 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.714491 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.714638 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.717520 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:58705: BlockingRecv error: recv got EOF from 127.11.118.193:58705 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":60,"threads_started":1}
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:09 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:09.901597 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:09.895467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:38573)
0929 18:41:09.895902 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:09.895915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:09.896058 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:09.896350 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:09.896353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:09.896560 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:09.896678 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.896684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.897451 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.897457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.898019 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:09.898027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.898594 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.898599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:09.898697 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.899293 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:09.899305 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:09.901419 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":143,"threads_started":1}
W20260929 18:41:09.901687 20911 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:09.895676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38573 (local address 127.11.118.254:34623)
0929 18:41:09.895857 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:09.895861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:09.895995 (+ 134us) server_negotiation.cc:407] Connection header received
0929 18:41:09.896171 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:09.896174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:09.896224 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:09.896323 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:09.896796 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.897334 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.898146 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:09.898454 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:09.898567 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:09.901553 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:38573: BlockingRecv error: recv got EOF from 127.11.118.194:38573 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:10.107049 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:10.097203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:49857)
0929 18:41:10.097478 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:10.097492 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:10.097588 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:10.097835 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:10.097837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:10.098013 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:10.098112 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.098117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.098898 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.098900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.099444 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.099451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.099871 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.099873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.099936 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.104803 (+ 4867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:10.104816 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:10.106881 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
W20260929 18:41:10.107199 20911 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:10.097391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:49857 (local address 127.11.118.254:34623)
0929 18:41:10.097521 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:10.097525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:10.097539 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:10.097676 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:10.097678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:10.097722 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:10.097795 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:10.098212 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.098802 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.099543 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.099770 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.099850 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.107046 (+ 7196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:49857: BlockingRecv error: recv got EOF from 127.11.118.195:49857 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:10.724548 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:10.718427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:49331)
0929 18:41:10.718812 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:10.718831 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:10.718977 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:10.719324 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:10.719327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:10.719658 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:10.719759 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.719765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.720534 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.720537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.721133 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.721141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.721641 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.721644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.721705 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.722197 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:10.722208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:10.724410 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
W20260929 18:41:10.724745 20915 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:10.718506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:49331 (local address 127.11.118.254:34623)
0929 18:41:10.718778 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:10.718783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:10.718917 (+ 134us) server_negotiation.cc:407] Connection header received
0929 18:41:10.719077 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:10.719081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:10.719168 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:10.719262 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:10.719870 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.720404 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.721256 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.721509 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.721597 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.724594 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:49331: BlockingRecv error: recv got EOF from 127.11.118.193:49331 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:10 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:10.908141 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:10.902620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:45245)
0929 18:41:10.902906 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:10.902924 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:10.903087 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:10.903319 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:10.903321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:10.903495 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:10.903591 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.903596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.904325 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.904327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.904882 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:10.904889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.905365 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.905367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:10.905430 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.905924 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:10.905935 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:10.908018 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":108,"threads_started":1}
W20260929 18:41:10.908319 20915 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:10.902656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:45245 (local address 127.11.118.254:34623)
0929 18:41:10.902816 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:10.902822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:10.903044 (+ 222us) server_negotiation.cc:407] Connection header received
0929 18:41:10.903183 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:10.903185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:10.903232 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:10.903306 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:10.903712 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.904229 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.905012 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:10.905253 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:10.905342 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:10.908182 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:45245: BlockingRecv error: recv got EOF from 127.11.118.194:45245 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:11.113708 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:11.107917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:35279)
0929 18:41:11.108200 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:11.108220 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:11.108325 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:11.108583 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:11.108585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:11.108773 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:11.108869 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.108874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.109680 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.109683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.110257 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.110264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.110778 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.110782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.110869 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.111447 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:11.111459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:11.113536 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":112,"threads_started":1}
W20260929 18:41:11.113822 20915 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:11.108050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:35279 (local address 127.11.118.254:34623)
0929 18:41:11.108194 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:11.108197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:11.108299 (+ 102us) server_negotiation.cc:407] Connection header received
0929 18:41:11.108405 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:11.108408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:11.108454 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:11.108543 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:11.109003 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.109574 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.110415 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.110656 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.110774 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.113696 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:35279: BlockingRecv error: recv got EOF from 127.11.118.195:35279 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:11.736830 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:11.725444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:44191)
0929 18:41:11.725727 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:11.725745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:11.725849 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:11.726079 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:11.726081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:11.726257 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:11.726383 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.726388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.727119 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.727122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.727722 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.727731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.728187 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.728190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.728254 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.734250 (+ 5996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:11.734266 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:11.736688 (+ 2422us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":71,"threads_started":1}
W20260929 18:41:11.737139 20919 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:11.725546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:44191 (local address 127.11.118.254:34623)
0929 18:41:11.725845 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:11.725849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:11.725862 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:11.725928 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:11.725930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:11.725971 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:11.726039 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:11.726480 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.727007 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.727837 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.728059 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.728177 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.736998 (+ 8821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:44191: BlockingRecv error: recv got EOF from 127.11.118.193:44191 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":130,"threads_started":1}
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:11 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:11.914955 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:11.909005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:50507)
0929 18:41:11.909281 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:11.909296 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:11.909396 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:11.909679 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:11.909681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:11.909856 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:11.909956 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.909961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.910851 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.910854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.911438 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:11.911446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.911948 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.911951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:11.912027 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.912548 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:11.912568 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:11.914791 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
W20260929 18:41:11.915112 20919 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:11.909147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:50507 (local address 127.11.118.254:34623)
0929 18:41:11.909312 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:11.909315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:11.909369 (+ 54us) server_negotiation.cc:407] Connection header received
0929 18:41:11.909477 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:11.909479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:11.909522 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:11.909616 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:11.910098 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.910724 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.911593 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:11.911821 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:11.911909 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:11.914957 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:50507: BlockingRecv error: recv got EOF from 127.11.118.194:50507 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:12.120512 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:12.114682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:35453)
0929 18:41:12.114947 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:12.114960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:12.115061 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:12.115313 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:12.115316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:12.115543 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:12.115645 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.115651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.116506 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.116510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.117113 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.117123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.117712 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.117716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.117810 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.118465 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:12.118483 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:12.120390 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":109,"threads_started":1}
W20260929 18:41:12.120712 20919 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:12.114759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:35453 (local address 127.11.118.254:34623)
0929 18:41:12.114905 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:12.114910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:12.115034 (+ 124us) server_negotiation.cc:407] Connection header received
0929 18:41:12.115132 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:12.115134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:12.115182 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:12.115252 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:12.115793 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.116374 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.117249 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.117578 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.117687 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.120536 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:35453: BlockingRecv error: recv got EOF from 127.11.118.195:35453 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:12.744074 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:12.737811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:32805)
0929 18:41:12.738171 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:12.738185 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:12.738316 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:12.738579 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:12.738581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:12.738759 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:12.738853 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.738857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.739746 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.739750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.740674 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.740686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.741156 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.741160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.741281 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.741886 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:12.741898 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:12.743950 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":170,"threads_started":1}
W20260929 18:41:12.744269 20923 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:12.737940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:32805 (local address 127.11.118.254:34623)
0929 18:41:12.738182 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:12.738186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:12.738308 (+ 122us) server_negotiation.cc:407] Connection header received
0929 18:41:12.738400 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:12.738403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:12.738448 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:12.738530 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:12.738954 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.739634 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.740794 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.741013 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.741098 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.744091 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:32805: BlockingRecv error: recv got EOF from 127.11.118.193:32805 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:12 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:12.921939 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:12.916116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:53561)
0929 18:41:12.916401 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:12.916414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:12.916509 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:12.916738 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:12.916740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:12.916924 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:12.917028 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.917033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.918047 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.918051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.918824 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:12.918838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.919414 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.919416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:12.919482 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.920043 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:12.920064 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:12.921823 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
W20260929 18:41:12.922158 20923 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:12.916290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:53561 (local address 127.11.118.254:34623)
0929 18:41:12.916429 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:12.916433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:12.916464 (+ 31us) server_negotiation.cc:407] Connection header received
0929 18:41:12.916584 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:12.916587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:12.916630 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:12.916700 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:12.917262 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.917930 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.918983 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:12.919300 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:12.919390 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:12.922034 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:53561: BlockingRecv error: recv got EOF from 127.11.118.194:53561 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:13.127147 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:13.121410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:53543)
0929 18:41:13.121704 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:13.121718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:13.121820 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:13.122049 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:13.122051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:13.122228 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:13.122399 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.122406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.123252 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.123255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.123974 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.123983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.124390 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.124414 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.124543 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.125098 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:13.125110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:13.127029 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":72,"threads_started":1}
W20260929 18:41:13.127300 20923 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:13.121508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:53543 (local address 127.11.118.254:34623)
0929 18:41:13.121703 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:13.121713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:13.121791 (+ 78us) server_negotiation.cc:407] Connection header received
0929 18:41:13.121895 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:13.121897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:13.121941 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:13.122010 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:13.122515 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.123143 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.124072 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.124285 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.124367 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.127182 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:53543: BlockingRecv error: recv got EOF from 127.11.118.195:53543 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:13.751271 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:13.745025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:46485)
0929 18:41:13.745327 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:13.745351 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:13.745463 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:13.745872 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:13.745875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:13.746087 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:13.746258 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.746265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.747151 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.747154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.747838 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.747849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.748418 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.748423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.748506 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.749109 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:13.749122 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:13.751150 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
W20260929 18:41:13.751454 20927 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:13.745096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:46485 (local address 127.11.118.254:34623)
0929 18:41:13.745390 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:13.745394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:13.745438 (+ 44us) server_negotiation.cc:407] Connection header received
0929 18:41:13.745635 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:13.745638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:13.745691 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:13.745797 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:13.746450 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.747007 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.747982 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.748280 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.748391 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.751311 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:46485: BlockingRecv error: recv got EOF from 127.11.118.193:46485 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:13 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:13.928776 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:13.922844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:43491)
0929 18:41:13.923124 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:13.923137 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:13.923238 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:13.923506 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:13.923508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:13.923759 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:13.923858 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.923864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.924650 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.924653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.925378 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:13.925388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.925868 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.925871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:13.925940 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.926644 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:13.926656 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:13.928653 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":108,"threads_started":1}
W20260929 18:41:13.929008 20927 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:13.922939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:43491 (local address 127.11.118.254:34623)
0929 18:41:13.923095 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:13.923099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:13.923207 (+ 108us) server_negotiation.cc:407] Connection header received
0929 18:41:13.923304 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:13.923306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:13.923350 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:13.923466 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:13.923974 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.924531 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.925515 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:13.925742 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:13.925831 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:13.928834 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:43491: BlockingRecv error: recv got EOF from 127.11.118.194:43491 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:14.133957 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:14.128156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:59615)
0929 18:41:14.128756 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:14.128770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:14.128931 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:14.129217 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:14.129220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:14.129430 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:14.129550 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.129556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.130378 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.130385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.130941 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.130949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.131407 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.131411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.131498 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.131999 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:14.132010 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:14.133785 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":87,"threads_started":1}
W20260929 18:41:14.134028 20927 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:14.128306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:59615 (local address 127.11.118.254:34623)
0929 18:41:14.128572 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:14.128577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:14.128874 (+ 297us) server_negotiation.cc:407] Connection header received
0929 18:41:14.129047 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:14.129050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:14.129105 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:14.129201 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:14.129677 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.130240 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.131067 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.131291 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.131404 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.133909 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:59615: BlockingRecv error: recv got EOF from 127.11.118.195:59615 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:14.757840 20930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:14.752173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:35825)
0929 18:41:14.752501 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:14.752517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:14.752611 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:14.752889 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:14.752891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:14.753087 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:14.753185 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.753190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.753999 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.754002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.754675 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.754685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.755127 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.755129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.755190 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.755756 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:14.755796 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:14.757720 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
W20260929 18:41:14.758057 20931 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:14.752282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:35825 (local address 127.11.118.254:34623)
0929 18:41:14.752564 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:14.752567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:14.752580 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:14.752697 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:14.752699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:14.752740 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:14.752825 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:14.753299 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.753893 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.754792 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.755029 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.755111 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.757899 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:35825: BlockingRecv error: recv got EOF from 127.11.118.193:35825 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":103,"threads_started":1}
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:14 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:14.935781 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:14.929795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:47513)
0929 18:41:14.930142 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:14.930154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:14.930334 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:14.930643 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:14.930645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:14.930862 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:14.930953 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.930960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.931718 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.931723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.932292 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:14.932301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.932784 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.932788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:14.932876 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.933413 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:14.933425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:14.935660 (+ 2235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":102,"threads_started":1}
W20260929 18:41:14.936070 20931 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:14.929875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:47513 (local address 127.11.118.254:34623)
0929 18:41:14.930039 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:14.930043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:14.930232 (+ 189us) server_negotiation.cc:407] Connection header received
0929 18:41:14.930448 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:14.930452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:14.930535 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:14.930616 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:14.931076 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.931603 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.932439 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:14.932668 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:14.932780 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:14.935940 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:47513: BlockingRecv error: recv got EOF from 127.11.118.194:47513 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:15.143651 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:15.134927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:49061)
0929 18:41:15.135262 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:15.135275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:15.135372 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:15.135652 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:15.135656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:15.135845 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:15.135938 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.135943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.136787 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.136793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.137369 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.137378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.137798 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.137801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.137865 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.141447 (+ 3582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:15.141464 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:15.143520 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":75,"threads_started":1}
W20260929 18:41:15.143837 20931 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:15.134951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:49061 (local address 127.11.118.254:34623)
0929 18:41:15.135104 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:15.135109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:15.135371 (+ 262us) server_negotiation.cc:407] Connection header received
0929 18:41:15.135443 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:15.135447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:15.135508 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:15.135593 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:15.136040 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.136636 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.137476 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.137689 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.137765 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.143699 (+ 5934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:49061: BlockingRecv error: recv got EOF from 127.11.118.195:49061 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:15.764907 20934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:15.758777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:37269)
0929 18:41:15.759064 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:15.759078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:15.759172 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:15.759571 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:15.759573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:15.759801 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:15.759957 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.759965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.761145 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.761150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.761744 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.761754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.762219 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.762222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.762358 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.762937 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:15.762949 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:15.764787 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
W20260929 18:41:15.765122 20935 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:15.758997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:37269 (local address 127.11.118.254:34623)
0929 18:41:15.759272 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:15.759276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:15.759291 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:15.759336 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:15.759339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:15.759389 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:15.759550 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:15.760111 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.760994 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.761875 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.762118 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.762200 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.764959 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:37269: BlockingRecv error: recv got EOF from 127.11.118.193:37269 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":111,"threads_started":1}
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:15 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:15.943380 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:15.936680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:51629)
0929 18:41:15.936970 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:15.936993 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:15.937096 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:15.937414 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:15.937417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:15.937596 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:15.937704 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.937710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.938740 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.938743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.939375 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:15.939384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.939922 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.939926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:15.939998 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.940657 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:15.940671 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:15.943203 (+ 2532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
W20260929 18:41:15.943462 20935 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:15.936788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:51629 (local address 127.11.118.254:34623)
0929 18:41:15.936961 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:15.936968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:15.937049 (+ 81us) server_negotiation.cc:407] Connection header received
0929 18:41:15.937183 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:15.937185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:15.937232 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:15.937315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:15.937850 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.938607 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.939555 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:15.939791 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:15.939884 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:15.943348 (+ 3464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:51629: BlockingRecv error: recv got EOF from 127.11.118.194:51629 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:16.150414 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:16.144588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:42485)
0929 18:41:16.144851 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:16.144872 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:16.144999 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:16.145332 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:16.145336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:16.145582 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:16.145708 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.145715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.146730 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.146733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.147320 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.147328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.147843 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.147845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.147908 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.148462 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:16.148483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:16.150241 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
W20260929 18:41:16.150607 20935 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:16.144629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:42485 (local address 127.11.118.254:34623)
0929 18:41:16.144813 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:16.144818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:16.144955 (+ 137us) server_negotiation.cc:407] Connection header received
0929 18:41:16.145113 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:16.145115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:16.145190 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:16.145279 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:16.145834 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.146632 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.147514 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.147742 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.147829 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.150443 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:42485: BlockingRecv error: recv got EOF from 127.11.118.195:42485 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:16.771662 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:16.765852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:33717)
0929 18:41:16.766140 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:16.766154 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:16.766252 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:16.766675 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:16.766677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:16.766861 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:16.766956 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.766961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.767733 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.767736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.768299 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.768307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.768744 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.768747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.768807 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.769561 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:16.769573 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:16.771543 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
W20260929 18:41:16.771881 20939 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:16.765955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:33717 (local address 127.11.118.254:34623)
0929 18:41:16.766188 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:16.766192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:16.766206 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:16.766417 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:16.766419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:16.766469 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:16.766590 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:16.767068 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.767615 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.768406 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.768643 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.768720 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.771717 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:33717: BlockingRecv error: recv got EOF from 127.11.118.193:33717 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":63,"threads_started":1}
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:16 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:16.950148 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:16.944179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:54651)
0929 18:41:16.944569 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:16.944581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:16.944722 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:16.944998 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:16.945001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:16.945211 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:16.945306 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.945315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.946129 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.946132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.946722 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:16.946731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.947194 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.947198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:16.947283 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.947916 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:16.947928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:16.949937 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":89,"threads_started":1}
W20260929 18:41:16.950208 20939 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:16.944313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:54651 (local address 127.11.118.254:34623)
0929 18:41:16.944493 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:16.944497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:16.944659 (+ 162us) server_negotiation.cc:407] Connection header received
0929 18:41:16.944822 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:16.944825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:16.944878 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:16.944979 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:16.945427 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.945982 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.946848 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:16.947078 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:16.947191 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:16.950094 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:54651: BlockingRecv error: recv got EOF from 127.11.118.194:54651 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:17.156608 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:17.151175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:35175)
0929 18:41:17.151474 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:17.151488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:17.151585 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:17.151827 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:17.151829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:17.152011 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:17.152104 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.152109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.152904 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.152907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.153470 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.153480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.153942 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.153946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.154015 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.154548 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:17.154560 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:17.156458 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
W20260929 18:41:17.156791 20939 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:17.151278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:35175 (local address 127.11.118.254:34623)
0929 18:41:17.151441 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:17.151445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:17.151560 (+ 115us) server_negotiation.cc:407] Connection header received
0929 18:41:17.151671 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:17.151673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:17.151718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:17.151786 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:17.152209 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.152752 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.153599 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.153815 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.153895 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.156662 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:35175: BlockingRecv error: recv got EOF from 127.11.118.195:35175 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:17.778637 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:17.772590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:58331)
0929 18:41:17.772906 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:17.772925 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:17.773042 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:17.773318 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:17.773320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:17.773528 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:17.773647 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.773653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.774570 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.774573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.775144 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.775153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.775680 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.775683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.775766 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.776272 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:17.776283 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:17.778499 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
W20260929 18:41:17.778959 20943 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:17.772676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:58331 (local address 127.11.118.254:34623)
0929 18:41:17.772922 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:17.772926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:17.773008 (+ 82us) server_negotiation.cc:407] Connection header received
0929 18:41:17.773127 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:17.773129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:17.773171 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:17.773249 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:17.773802 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.774440 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.775264 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.775520 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.775643 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.778660 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:58331: BlockingRecv error: recv got EOF from 127.11.118.193:58331 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:17 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:17.956843 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:17.951059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:44619)
0929 18:41:17.951397 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:17.951414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:17.951562 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:17.951874 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:17.951877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:17.952129 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:17.952260 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.952269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.953102 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.953105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.953677 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:17.953685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.954175 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.954177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:17.954240 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.954788 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:17.954799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:17.956716 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":99,"threads_started":1}
W20260929 18:41:17.957038 20943 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:17.951173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:44619 (local address 127.11.118.254:34623)
0929 18:41:17.951302 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:17.951309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:17.951501 (+ 192us) server_negotiation.cc:407] Connection header received
0929 18:41:17.951676 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:17.951679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:17.951741 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:17.951827 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:17.952451 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.952990 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.953837 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:17.954064 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:17.954161 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:17.956884 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:44619: BlockingRecv error: recv got EOF from 127.11.118.194:44619 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:18.163028 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:18.157497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:41025)
0929 18:41:18.157760 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:18.157777 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:18.157879 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:18.158204 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:18.158207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:18.158435 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:18.158526 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.158532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.159279 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.159285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.159870 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.159878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.160340 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.160343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.160433 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.160967 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:18.160979 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:18.162910 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
W20260929 18:41:18.163203 20943 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:18.157696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:41025 (local address 127.11.118.254:34623)
0929 18:41:18.157826 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:18.157831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:18.157846 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:18.158029 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:18.158032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:18.158087 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:18.158181 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:18.158648 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.159165 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.160000 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.160227 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.160338 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.163044 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:41025: BlockingRecv error: recv got EOF from 127.11.118.195:41025 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:18.785142 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:18.779480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:52175)
0929 18:41:18.779832 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:18.779853 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:18.779949 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:18.780223 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:18.780225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:18.780458 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:18.780554 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.780560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.781283 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.781286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.781892 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.781900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.782532 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.782535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.782614 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.783152 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:18.783164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:18.785027 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":89,"threads_started":1}
W20260929 18:41:18.785302 20947 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:18.779632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:52175 (local address 127.11.118.254:34623)
0929 18:41:18.779886 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:18.779889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:18.779904 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:41:18.780059 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:18.780061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:18.780105 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:18.780184 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:18.780657 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.781188 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.781995 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.782220 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.782321 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.785164 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:52175: BlockingRecv error: recv got EOF from 127.11.118.193:52175 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:18 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:18.963575 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:18.957643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:36145)
0929 18:41:18.957990 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:18.958004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:18.958145 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:18.958462 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:18.958465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:18.958675 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:18.958768 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.958774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.959549 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.959554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.960122 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:18.960130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.960589 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.960593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:18.960680 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.961238 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:18.961250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:18.963381 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
W20260929 18:41:18.963639 20947 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:18.957775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:36145 (local address 127.11.118.254:34623)
0929 18:41:18.957945 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:18.957950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:18.958081 (+ 131us) server_negotiation.cc:407] Connection header received
0929 18:41:18.958247 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:18.958251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:18.958328 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:18.958436 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:18.958910 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.959433 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.960242 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:18.960473 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:18.960582 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:18.963526 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:36145: BlockingRecv error: recv got EOF from 127.11.118.194:36145 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:19.169269 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:19.163850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:40343)
0929 18:41:19.164137 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:19.164150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:19.164250 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:19.164490 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:19.164492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:19.164670 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:19.164764 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.164769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.165502 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.165505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.166052 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.166060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.166567 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.166570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.166640 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.167222 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:19.167234 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:19.169151 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
W20260929 18:41:19.169456 20947 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:19.163933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:40343 (local address 127.11.118.254:34623)
0929 18:41:19.164118 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:19.164122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:19.164222 (+ 100us) server_negotiation.cc:407] Connection header received
0929 18:41:19.164330 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:19.164332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:19.164376 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:19.164443 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:19.164874 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.165375 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.166186 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.166436 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.166526 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.169288 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:40343: BlockingRecv error: recv got EOF from 127.11.118.195:40343 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:19.791718 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:19.786006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:42481)
0929 18:41:19.786425 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:19.786445 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:19.786570 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:19.786806 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:19.786809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:19.786982 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:19.787078 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.787083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.787838 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.787842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.788406 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.788414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.788898 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.788901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.788969 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.789481 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:19.789493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:19.791596 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
W20260929 18:41:19.791965 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:19.786142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:42481 (local address 127.11.118.254:34623)
0929 18:41:19.786401 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:19.786405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:19.786565 (+ 160us) server_negotiation.cc:407] Connection header received
0929 18:41:19.786640 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:19.786642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:19.786688 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:19.786774 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:19.787182 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.787721 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.788551 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.788781 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.788871 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.791802 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:42481: BlockingRecv error: recv got EOF from 127.11.118.193:42481 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":56,"threads_started":1}
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:19 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:19.970065 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:19.964447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:35309)
0929 18:41:19.964760 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:19.964774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:19.964873 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:19.965141 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:19.965144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:19.965391 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:19.965495 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.965502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.966264 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.966298 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.966862 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:19.966870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.967537 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.967540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:19.967604 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.968127 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:19.968138 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:19.969947 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
W20260929 18:41:19.970233 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:19.964573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:35309 (local address 127.11.118.254:34623)
0929 18:41:19.964761 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:19.964764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:19.964844 (+ 80us) server_negotiation.cc:407] Connection header received
0929 18:41:19.964957 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:19.964959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:19.965003 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:19.965071 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:19.965604 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.966136 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.967017 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:19.967263 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:19.967544 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:19.970111 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:35309: BlockingRecv error: recv got EOF from 127.11.118.194:35309 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:20.180521 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:20.170129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:33567)
0929 18:41:20.170422 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:20.170444 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:20.170573 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:20.170852 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:20.170855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:20.171091 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:20.171187 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.171192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.171995 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.171998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.172576 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.172583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.173052 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.173056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.173145 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.178026 (+ 4881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:20.178040 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:20.180396 (+ 2356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
W20260929 18:41:20.180719 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:20.170214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:33567 (local address 127.11.118.254:34623)
0929 18:41:20.170381 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:20.170386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:20.170517 (+ 131us) server_negotiation.cc:407] Connection header received
0929 18:41:20.170679 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:20.170683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:20.170733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:20.170828 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:20.171306 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.171872 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.172695 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.172939 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.173049 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.180583 (+ 7534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:33567: BlockingRecv error: recv got EOF from 127.11.118.195:33567 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:20.798476 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:20.792692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:58947)
0929 18:41:20.792992 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:20.793005 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:20.793100 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:20.793345 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:20.793347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:20.793528 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:20.793624 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.793629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.794483 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.794486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.795236 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.795247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.795862 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.795866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.795979 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.796538 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:20.796550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:20.798359 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":117,"threads_started":1}
W20260929 18:41:20.798695 20955 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:20.792851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:58947 (local address 127.11.118.254:34623)
0929 18:41:20.793111 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:20.793114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:20.793131 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:41:20.793173 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:20.793175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:20.793216 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:20.793306 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:20.793750 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.794368 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.795350 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.795713 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.795814 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.798522 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:58947: BlockingRecv error: recv got EOF from 127.11.118.193:58947 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:20 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:20.977027 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:20.971153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:59015)
0929 18:41:20.971512 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:20.971526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:20.971631 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:20.971906 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:20.971908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:20.972099 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:20.972195 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.972200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.973039 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.973041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.973905 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:20.973917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.974439 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.974442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:20.974544 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.975097 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:20.975117 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:20.976899 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
W20260929 18:41:20.977219 20955 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:20.971352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:59015 (local address 127.11.118.254:34623)
0929 18:41:20.971548 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:20.971552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:20.971584 (+ 32us) server_negotiation.cc:407] Connection header received
0929 18:41:20.971725 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:20.971727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:20.971770 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:20.971843 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:20.972308 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.972903 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.974027 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:20.974250 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:20.974408 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:20.977050 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:59015: BlockingRecv error: recv got EOF from 127.11.118.194:59015 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:21.187112 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:21.181318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:56959)
0929 18:41:21.181623 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:21.181636 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:21.181726 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:21.181961 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:21.181963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:21.182146 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:21.182240 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.182246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.183091 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.183095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.183713 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.183723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.184232 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.184234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.184299 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.184849 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:21.184860 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:21.186975 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
W20260929 18:41:21.187299 20955 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:21.181523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:56959 (local address 127.11.118.254:34623)
0929 18:41:21.181679 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:21.181683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:21.181696 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:41:21.181817 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:21.181820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:21.181863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:21.181925 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:21.182427 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.182984 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.183901 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.184126 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.184217 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.187132 (+ 2915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:56959: BlockingRecv error: recv got EOF from 127.11.118.195:56959 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:21.805339 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:21.799504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:44945)
0929 18:41:21.799785 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:21.799808 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:21.799964 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:21.800306 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:21.800309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:21.800532 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:21.800628 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.800634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.801685 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.801688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.802319 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.802328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.802746 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.802749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.802819 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.803322 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:21.803333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:21.805212 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":116,"threads_started":1}
W20260929 18:41:21.805578 20959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:21.799648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:44945 (local address 127.11.118.254:34623)
0929 18:41:21.799995 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:21.800024 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:21.800042 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:41:21.800092 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:21.800096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:21.800146 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:21.800270 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:21.800757 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.801555 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.802424 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.802641 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.802720 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.805439 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:44945: BlockingRecv error: recv got EOF from 127.11.118.193:44945 (error 108)
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":105,"threads_started":1}
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:21 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:21.984119 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:21.977929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:33013)
0929 18:41:21.978264 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:21.978314 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:21.978464 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:21.978738 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:21.978741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:21.978973 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:21.979068 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.979074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.979843 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.979848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.980452 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:21.980461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.980939 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.980943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:21.981030 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.981567 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:21.981578 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:21.983936 (+ 2358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":99,"threads_started":1}
W20260929 18:41:21.984155 20959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:21.978008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:33013 (local address 127.11.118.254:34623)
0929 18:41:21.978229 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:21.978234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:21.978428 (+ 194us) server_negotiation.cc:407] Connection header received
0929 18:41:21.978572 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:21.978575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:21.978627 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:21.978717 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:21.979198 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.979729 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.980580 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:21.980811 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:21.980925 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:21.984054 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:33013: BlockingRecv error: recv got EOF from 127.11.118.194:33013 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.195@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:22.194684 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:22.187894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.195:41555)
0929 18:41:22.188136 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:22.188152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:22.188271 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:22.188501 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:22.188504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:22.188727 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:22.188838 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.188844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.189839 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.189843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.190589 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.190607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.191227 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.191233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.191356 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.191994 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:22.192013 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:22.194487 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
W20260929 18:41:22.194746 20959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:22.187942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:41555 (local address 127.11.118.254:34623)
0929 18:41:22.188182 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:22.188188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:22.188204 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:41:22.188358 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:22.188361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:22.188411 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:22.188484 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:22.188945 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.189720 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.190735 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.191070 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.191181 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.194628 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.195:41555: BlockingRecv error: recv got EOF from 127.11.118.195:41555 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.193@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:22.812108 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:22.806362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.193:55755)
0929 18:41:22.806709 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:22.806727 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:22.806854 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:22.807107 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:22.807109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:22.807297 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:22.807389 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.807394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.808182 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.808185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.808779 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.808787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.809246 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.809249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.809349 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.809941 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:22.809953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:22.811918 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":93,"threads_started":1}
W20260929 18:41:22.812184 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:22.806480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:55755 (local address 127.11.118.254:34623)
0929 18:41:22.806725 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:22.806729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:22.806797 (+ 68us) server_negotiation.cc:407] Connection header received
0929 18:41:22.806949 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:22.806951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:22.806996 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:22.807070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:22.807525 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.808073 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.808908 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.809139 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.809223 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.812055 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.193:55755: BlockingRecv error: recv got EOF from 127.11.118.193:55755 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":61,"threads_started":1}
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 18:41:22 dist-test-slave-cgv7 krb5kdc[19904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.118.194@KRBTEST.COM for oryx/127.11.118.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 18:41:22.991055 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:22.985217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34623 (local address 127.11.118.194:54533)
0929 18:41:22.985505 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:22.985519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:22.985664 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:22.985995 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:22.985998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:22.986249 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:22.986406 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.986415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.987216 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.987219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.987801 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:22.987809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.988306 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.988310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:22.988390 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.988901 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:22.988913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:22.990923 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.118.254:34623: Server oryx/127.11.118.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
W20260929 18:41:22.991209 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:22.985305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:54533 (local address 127.11.118.254:34623)
0929 18:41:22.985497 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:22.985503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:22.985630 (+ 127us) server_negotiation.cc:407] Connection header received
0929 18:41:22.985780 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:22.985783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:22.985841 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:22.985934 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:22.986557 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.987106 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.987928 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:22.988157 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:22.988241 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:22.991081 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.118.194:54533: BlockingRecv error: recv got EOF from 127.11.118.194:54533 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 18:41:23.080268 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20487
I20260929 18:41:23.086128 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20639
I20260929 18:41:23.091598 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20772
I20260929 18:41:23.096920 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20417
2026-09-29T18:41:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18062 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T18:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:23Z Disabled control of system clock
I20260929 18:41:23.128594 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39655
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 18:41:23.242255 20968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:23.242570 20968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:23.242605 20968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:23.246798 20968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:23.246867 20968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:23.246883 20968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:23.246901 20968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:23.246915 20968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:23.251662 20968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:23.252772 20968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:23.254210 20968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:23.263348 20973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.263486 20968 server_base.cc:1034] running on GCE node
W20260929 18:41:23.263332 20974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.263332 20976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.264257 20968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:23.265431 20968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:23.266630 20968 hybrid_clock.cc:648] HybridClock initialized: now 1790707283266616 us; error 33 us; skew 500 ppm
I20260929 18:41:23.268576 20968 webserver.cc:460] Webserver started at http://127.11.118.254:34873/ using document root <none> and password file <none>
I20260929 18:41:23.269248 20968 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:23.269342 20968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:23.269567 20968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:23.271754 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "f782537714d048a998daa4f5cd68043d"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "0132c71264aa8d4dd68c5a7a35e241a1"
server_key_iv: "d1ee0aa12649896131c5bcc9fbb6f62b"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.272362 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "f782537714d048a998daa4f5cd68043d"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "0132c71264aa8d4dd68c5a7a35e241a1"
server_key_iv: "d1ee0aa12649896131c5bcc9fbb6f62b"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.276270 20968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:41:23.278955 20982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.280280 20968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:41:23.280432 20968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f782537714d048a998daa4f5cd68043d"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "0132c71264aa8d4dd68c5a7a35e241a1"
server_key_iv: "d1ee0aa12649896131c5bcc9fbb6f62b"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.280561 20968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:23.298765 20968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:23.299561 20968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:23.299787 20968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:23.306857 20968 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39655
I20260929 18:41:23.306864 21034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39655 every 8 connection(s)
I20260929 18:41:23.307938 20968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:23.311509 21035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:23.315260 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 20968
I20260929 18:41:23.315428 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:41:23.315748 11739 external_mini_cluster.cc:1428] Setting key 2b18ed384e80a767fca670501fc86b8b
I20260929 18:41:23.317962 21035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d: Bootstrap starting.
I20260929 18:41:23.320369 21035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:23.321159 21035 log.cc:824] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:23.323292 21035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d: No bootstrap required, opened a new log
I20260929 18:41:23.325649 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.317292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.11.118.254:39655)
0929 18:41:23.318006 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:23.318023 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:23.318097 (+ 74us) server_negotiation.cc:407] Connection header received
0929 18:41:23.318924 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:23.318944 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:23.319366 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:23.319723 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:23.320189 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.321281 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.322213 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.322775 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.322962 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.323854 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:23.323860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:23.323874 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:23.324292 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:23.324348 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:23.324368 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:23.324484 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:23.324638 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 18:41:23.324911 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":136,"threads_started":1}
I20260929 18:41:23.327760 21035 raft_consensus.cc:348] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } }
I20260929 18:41:23.328080 21035 raft_consensus.cc:374] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:23.328156 21035 raft_consensus.cc:729] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f782537714d048a998daa4f5cd68043d, State: Initialized, Role: FOLLOWER
I20260929 18:41:23.328594 21035 consensus_queue.cc:260] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } }
I20260929 18:41:23.328716 21035 raft_consensus.cc:388] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:23.328788 21035 raft_consensus.cc:482] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:23.328896 21035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:23.329883 21035 raft_consensus.cc:504] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } }
I20260929 18:41:23.330242 21035 leader_election.cc:302] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f782537714d048a998daa4f5cd68043d; no voters:
I20260929 18:41:23.330634 21035 leader_election.cc:288] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:41:23.330811 21040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:23.331069 21040 raft_consensus.cc:686] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [term 1 LEADER]: Becoming Leader. State: Replica: f782537714d048a998daa4f5cd68043d, State: Running, Role: LEADER
I20260929 18:41:23.331530 21040 consensus_queue.cc:237] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } }
I20260929 18:41:23.332000 21035 sys_catalog.cc:564] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:23.333284 21041 sys_catalog.cc:455] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f782537714d048a998daa4f5cd68043d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } } }
I20260929 18:41:23.333284 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f782537714d048a998daa4f5cd68043d. Latest consensus state: current_term: 1 leader_uuid: "f782537714d048a998daa4f5cd68043d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f782537714d048a998daa4f5cd68043d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39655 } } }
I20260929 18:41:23.333432 21041 sys_catalog.cc:458] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:23.333432 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:23.333873 21049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:23.337118 21049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:23.342724 21049 catalog_manager.cc:1347] Generated new cluster ID: c039fcd52e534db7bf647a1ca3a886ff
I20260929 18:41:23.342814 21049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:23.351442 21049 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:41:23.352416 21049 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:23.356501 21049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f782537714d048a998daa4f5cd68043d: Generated new TSK 0
I20260929 18:41:23.357208 21049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:41:23.362040 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--builtin_ntp_servers=127.11.118.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 18:41:23.480561 21059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:23.480831 21059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:23.480870 21059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:23.485212 21059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:23.485275 21059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:23.485365 21059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:23.490389 21059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:23.491623 21059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:23.493119 21059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:23.502900 21064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.502900 21065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.502925 21067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.503093 21059 server_base.cc:1034] running on GCE node
I20260929 18:41:23.503619 21059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:23.504212 21059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:23.505373 21059 hybrid_clock.cc:648] HybridClock initialized: now 1790707283505352 us; error 35 us; skew 500 ppm
I20260929 18:41:23.507424 21059 webserver.cc:460] Webserver started at http://127.11.118.193:41257/ using document root <none> and password file <none>
I20260929 18:41:23.508059 21059 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:23.508122 21059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:23.508347 21059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:23.510560 21059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "97fbab701cf8425988a28185be7f85d8"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "61450a604774e05015cd6bd95e033a22"
server_key_iv: "84f0e582896edd9a77ef24d03cbe9a47"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.511133 21059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "97fbab701cf8425988a28185be7f85d8"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "61450a604774e05015cd6bd95e033a22"
server_key_iv: "84f0e582896edd9a77ef24d03cbe9a47"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.514992 21059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:41:23.517607 21073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.519004 21059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 18:41:23.519145 21059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "97fbab701cf8425988a28185be7f85d8"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "61450a604774e05015cd6bd95e033a22"
server_key_iv: "84f0e582896edd9a77ef24d03cbe9a47"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.519264 21059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:23.530043 21059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:23.530853 21059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:23.531073 21059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:23.531729 21059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:23.532873 21059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:23.532944 21059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.533013 21059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:23.533061 21059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.541980 21059 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:41919
I20260929 18:41:23.541999 21186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:41919 every 8 connection(s)
I20260929 18:41:23.543006 21059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:23.549424 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21059
I20260929 18:41:23.549580 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:41:23.549961 11739 external_mini_cluster.cc:1428] Setting key 4b6f204a6d5eca7a3fe741f374291008
I20260929 18:41:23.552256 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--builtin_ntp_servers=127.11.118.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 18:41:23.553464 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.545025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:45081 (local address 127.11.118.254:39655)
0929 18:41:23.545196 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:23.545199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:23.546374 (+ 1175us) server_negotiation.cc:407] Connection header received
0929 18:41:23.547875 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:23.547878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:23.547937 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:23.548030 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:23.548999 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.549612 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.551199 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.551555 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.551676 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.552942 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:23.552946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:23.552948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:23.552963 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:23.552994 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:23.553003 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:23.553084 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:23.553251 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 18:41:23.553317 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 18:41:23.554484 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.545476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39655 (local address 127.11.118.193:45081)
0929 18:41:23.546186 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:23.546228 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:23.547633 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:23.548234 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:23.548241 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:23.548335 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:23.548846 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.548859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.549759 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.549765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.551059 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.551069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.551746 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.551750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.551937 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.552655 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:23.552657 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:23.552699 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:23.552770 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:23.553086 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:23.553097 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:23.553105 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 18:41:23.553251 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 18:41:23.553580 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260929 18:41:23.555821 21187 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39655
I20260929 18:41:23.556100 21187 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:23.556733 21187 heartbeater.cc:511] Master 127.11.118.254:39655 requested a full tablet report, sending...
I20260929 18:41:23.558627 20999 ts_manager.cc:194] Registered new tserver with Master: 97fbab701cf8425988a28185be7f85d8 (127.11.118.193:41919)
I20260929 18:41:23.560513 20999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.193:45081
W20260929 18:41:23.672415 21190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:23.672822 21190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:23.672906 21190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:23.678799 21190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:23.678939 21190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:23.679049 21190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:23.685145 21190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:23.686234 21190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:23.687729 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:23.697515 21196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.697480 21195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.697503 21198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.697856 21190 server_base.cc:1034] running on GCE node
I20260929 18:41:23.698331 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:23.699049 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:23.700238 21190 hybrid_clock.cc:648] HybridClock initialized: now 1790707283700196 us; error 61 us; skew 500 ppm
I20260929 18:41:23.702136 21190 webserver.cc:460] Webserver started at http://127.11.118.194:41571/ using document root <none> and password file <none>
I20260929 18:41:23.702787 21190 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:23.702883 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:23.703104 21190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:23.705163 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "ab9413a9298449529b7f4318906ec32a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "ffd935f5468b8701c4c3e7016108cd26"
server_key_iv: "00491a636d7ce6c277b76b42afc66067"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.705747 21190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "ab9413a9298449529b7f4318906ec32a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "ffd935f5468b8701c4c3e7016108cd26"
server_key_iv: "00491a636d7ce6c277b76b42afc66067"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.709544 21190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:41:23.712365 21204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.713624 21190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:23.713765 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "ab9413a9298449529b7f4318906ec32a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "ffd935f5468b8701c4c3e7016108cd26"
server_key_iv: "00491a636d7ce6c277b76b42afc66067"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.713892 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:23.744082 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:23.744834 21190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:23.745054 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:23.745671 21190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:23.746716 21190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:23.746793 21190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.746866 21190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:23.746917 21190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.756312 21190 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:35267
I20260929 18:41:23.756330 21317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:35267 every 8 connection(s)
I20260929 18:41:23.757278 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:23.760203 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21190
I20260929 18:41:23.760322 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:41:23.760632 11739 external_mini_cluster.cc:1428] Setting key d5f31fdf6ca1ad2beee9cd2b4b22e70c
I20260929 18:41:23.762959 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--builtin_ntp_servers=127.11.118.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 18:41:23.768457 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.759313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:58219 (local address 127.11.118.254:39655)
0929 18:41:23.759491 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:23.759495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:23.760772 (+ 1277us) server_negotiation.cc:407] Connection header received
0929 18:41:23.762548 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:23.762551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:23.762603 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:23.762690 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:23.763690 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.764488 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.766056 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.766500 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.766627 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.767824 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:23.767832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:23.767836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:23.767855 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:23.767881 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:23.767895 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:23.768015 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:23.768155 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 18:41:23.768235 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 18:41:23.769224 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.759725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39655 (local address 127.11.118.194:58219)
0929 18:41:23.760590 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:23.760633 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:23.762324 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:23.762851 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:23.762858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:23.762941 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:23.763548 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.763562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.764623 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.764629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.765928 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.765937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.766650 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.766652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.766886 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.767548 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:23.767550 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:23.767590 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:23.767656 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:23.768012 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:23.768019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:23.768023 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 18:41:23.768119 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 18:41:23.768352 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":156,"threads_started":1}
I20260929 18:41:23.770536 21318 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39655
I20260929 18:41:23.770840 21318 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:23.771569 21318 heartbeater.cc:511] Master 127.11.118.254:39655 requested a full tablet report, sending...
I20260929 18:41:23.772866 20999 ts_manager.cc:194] Registered new tserver with Master: ab9413a9298449529b7f4318906ec32a (127.11.118.194:35267)
I20260929 18:41:23.773483 20999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.194:58219
W20260929 18:41:23.878871 21321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:23.879161 21321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:23.879231 21321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:23.883433 21321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:23.883526 21321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:23.883627 21321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:23.888767 21321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:23.889927 21321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:23.891396 21321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:23.901616 21327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:23.901626 21326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.901799 21321 server_base.cc:1034] running on GCE node
W20260929 18:41:23.901616 21329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:23.902376 21321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:23.902948 21321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:23.904136 21321 hybrid_clock.cc:648] HybridClock initialized: now 1790707283904120 us; error 29 us; skew 500 ppm
I20260929 18:41:23.905848 21321 webserver.cc:460] Webserver started at http://127.11.118.195:39349/ using document root <none> and password file <none>
I20260929 18:41:23.906471 21321 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:23.906533 21321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:23.906778 21321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:23.908742 21321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "3098f096ec4f47d9ad6c7d3217dea20a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "e0c8f1d1b231596bdca606f59d4a34fe"
server_key_iv: "c70e2f847ece09c8cbbf632358960436"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.909282 21321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "3098f096ec4f47d9ad6c7d3217dea20a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "e0c8f1d1b231596bdca606f59d4a34fe"
server_key_iv: "c70e2f847ece09c8cbbf632358960436"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.912858 21321 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260929 18:41:23.915409 21335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.916658 21321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:41:23.916800 21321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "3098f096ec4f47d9ad6c7d3217dea20a"
format_stamp: "Formatted at 2026-09-29 18:41:23 on dist-test-slave-cgv7"
server_key: "e0c8f1d1b231596bdca606f59d4a34fe"
server_key_iv: "c70e2f847ece09c8cbbf632358960436"
server_key_version: "encryptionkey@0"
I20260929 18:41:23.916975 21321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:23.951699 21321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:23.952466 21321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:23.952675 21321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:23.953270 21321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:23.954246 21321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:23.954339 21321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.954412 21321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:23.954470 21321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:23.964075 21321 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:33763
I20260929 18:41:23.964099 21448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:33763 every 8 connection(s)
I20260929 18:41:23.965003 21321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:23.970371 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21321
I20260929 18:41:23.970464 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:41:23.970729 11739 external_mini_cluster.cc:1428] Setting key cae2dbfb981b7341f68c2cdfb7601ed4
I20260929 18:41:23.974838 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.966955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:40751 (local address 127.11.118.254:39655)
0929 18:41:23.967097 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:23.967101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:23.968160 (+ 1059us) server_negotiation.cc:407] Connection header received
0929 18:41:23.969564 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:23.969567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:23.969622 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:23.969733 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:23.970752 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.971418 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.973060 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.973276 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.973355 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.974423 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:23.974427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:23.974429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:23.974442 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:23.974462 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:23.974470 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:23.974527 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:23.974675 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 18:41:23.974725 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:41:23.975775 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:23.967374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39655 (local address 127.11.118.195:40751)
0929 18:41:23.968016 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:23.968056 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:23.969351 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:23.969898 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:23.969905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:23.969989 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:23.970615 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.970630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.971569 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.971572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.972957 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:23.972966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:23.973431 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:23.973433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:23.973571 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:23.974139 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:23.974141 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:23.974180 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:23.974245 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:23.974548 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:23.974555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:23.974560 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:41:23.974660 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 18:41:23.974917 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260929 18:41:23.976857 21449 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39655
I20260929 18:41:23.977101 21449 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:23.977623 21449 heartbeater.cc:511] Master 127.11.118.254:39655 requested a full tablet report, sending...
I20260929 18:41:23.978653 20999 ts_manager.cc:194] Registered new tserver with Master: 3098f096ec4f47d9ad6c7d3217dea20a (127.11.118.195:33763)
I20260929 18:41:23.979187 20999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.195:40751
I20260929 18:41:23.984647 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 18:41:23.987574 21038 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:23.986626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.11.118.254:39655)
0929 18:41:23.986796 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:23.986801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:23.986943 (+ 142us) server_negotiation.cc:407] Connection header received
0929 18:41:23.987050 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:23.987053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:23.987094 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:23.987154 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:23.987389 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57978: BlockingRecv error: recv got EOF from 127.0.0.1:57978 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:41:23.987627 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21059
I20260929 18:41:23.993269 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21190
I20260929 18:41:23.998353 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21321
I20260929 18:41:24.003695 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 20968
2026-09-29T18:41:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (907 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T18:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:24Z Disabled control of system clock
I20260929 18:41:24.034678 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39245
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 18:41:24.147516 21460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:24.147873 21460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:24.147956 21460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:24.152078 21460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:24.152186 21460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:24.152235 21460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:24.152314 21460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:24.152354 21460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:24.157732 21460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:24.158989 21460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:24.160466 21460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:24.169646 21466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.169655 21465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.169759 21460 server_base.cc:1034] running on GCE node
W20260929 18:41:24.169674 21468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.170619 21460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:24.171777 21460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:24.173003 21460 hybrid_clock.cc:648] HybridClock initialized: now 1790707284172984 us; error 31 us; skew 500 ppm
I20260929 18:41:24.175410 21460 webserver.cc:460] Webserver started at http://127.11.118.254:39067/ using document root <none> and password file <none>
I20260929 18:41:24.176064 21460 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:24.176163 21460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:24.176415 21460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:24.178785 21460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "ab8c9902d3f04a8fabfe844d24c845c3"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "bdd0edc56f7353f844b5a58d29ce9944"
server_key_iv: "8a89886883f661c53abb220a5aee4926"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.179416 21460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "ab8c9902d3f04a8fabfe844d24c845c3"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "bdd0edc56f7353f844b5a58d29ce9944"
server_key_iv: "8a89886883f661c53abb220a5aee4926"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.183535 21460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260929 18:41:24.186209 21474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.187654 21460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 18:41:24.187824 21460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "ab8c9902d3f04a8fabfe844d24c845c3"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "bdd0edc56f7353f844b5a58d29ce9944"
server_key_iv: "8a89886883f661c53abb220a5aee4926"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.187961 21460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:24.192986 21460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:24.193675 21460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:24.193883 21460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:24.201241 21460 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39245
I20260929 18:41:24.201256 21526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39245 every 8 connection(s)
I20260929 18:41:24.202373 21460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:24.205838 21527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:24.210714 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21460
I20260929 18:41:24.210891 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:41:24.211205 11739 external_mini_cluster.cc:1428] Setting key 97fac7ef455979d26e9f8fa703e4b36e
I20260929 18:41:24.212255 21527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3: Bootstrap starting.
I20260929 18:41:24.214802 21527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:24.215745 21527 log.cc:824] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:24.217300 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.212853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55328 (local address 127.11.118.254:39245)
0929 18:41:24.213510 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:24.213526 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:24.213561 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:41:24.214212 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:24.214229 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:24.214678 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:24.215046 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:24.215900 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:24.215908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:24.215931 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:24.215943 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:24.216002 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:24.216020 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:24.216109 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:24.216278 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 18:41:24.216576 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260929 18:41:24.217919 21527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3: No bootstrap required, opened a new log
I20260929 18:41:24.221455 21527 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } }
I20260929 18:41:24.221719 21527 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:24.221807 21527 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8c9902d3f04a8fabfe844d24c845c3, State: Initialized, Role: FOLLOWER
I20260929 18:41:24.222236 21527 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } }
I20260929 18:41:24.222457 21527 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:24.222544 21527 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:24.222644 21527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:24.223598 21527 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } }
I20260929 18:41:24.223953 21527 leader_election.cc:302] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab8c9902d3f04a8fabfe844d24c845c3; no voters:
I20260929 18:41:24.224314 21527 leader_election.cc:288] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:41:24.224395 21532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:24.224596 21532 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [term 1 LEADER]: Becoming Leader. State: Replica: ab8c9902d3f04a8fabfe844d24c845c3, State: Running, Role: LEADER
I20260929 18:41:24.224961 21532 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } }
I20260929 18:41:24.225683 21527 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:24.226833 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } } }
I20260929 18:41:24.226845 21534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab8c9902d3f04a8fabfe844d24c845c3. Latest consensus state: current_term: 1 leader_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8c9902d3f04a8fabfe844d24c845c3" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39245 } } }
I20260929 18:41:24.227171 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:24.227205 21534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3 [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:24.228097 21541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:24.230916 21541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:24.237403 21541 catalog_manager.cc:1347] Generated new cluster ID: 98c04e7f3a3d45418a805898117c02f6
I20260929 18:41:24.237488 21541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:24.247682 21541 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:41:24.248077 21541 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:24.255956 21541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab8c9902d3f04a8fabfe844d24c845c3: Generated new TSK 0
I20260929 18:41:24.256729 21541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:41:24.264851 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--builtin_ntp_servers=127.11.118.212:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 18:41:24.386190 21551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:24.386575 21551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:24.386649 21551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:24.390982 21551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:24.391036 21551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:24.391125 21551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:24.396849 21551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:24.398036 21551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:24.399533 21551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:24.409490 21559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.409495 21557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.409495 21556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.409808 21551 server_base.cc:1034] running on GCE node
I20260929 18:41:24.410244 21551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:24.410914 21551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:24.412122 21551 hybrid_clock.cc:648] HybridClock initialized: now 1790707284412106 us; error 38 us; skew 500 ppm
I20260929 18:41:24.414198 21551 webserver.cc:460] Webserver started at http://127.11.118.193:44365/ using document root <none> and password file <none>
I20260929 18:41:24.414824 21551 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:24.414930 21551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:24.415169 21551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:24.417454 21551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "8f27e805fd534c9ca0099e2fd4f31a2c"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "4bda7a3292667b19f8ed54cf1aff7ba8"
server_key_iv: "3a81d7553abef74ab804ea7ce87ef3a9"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.418056 21551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "8f27e805fd534c9ca0099e2fd4f31a2c"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "4bda7a3292667b19f8ed54cf1aff7ba8"
server_key_iv: "3a81d7553abef74ab804ea7ce87ef3a9"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.421931 21551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 18:41:24.424731 21565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.426146 21551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:41:24.426322 21551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "8f27e805fd534c9ca0099e2fd4f31a2c"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "4bda7a3292667b19f8ed54cf1aff7ba8"
server_key_iv: "3a81d7553abef74ab804ea7ce87ef3a9"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.426483 21551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:24.431470 21551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:24.432142 21551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:24.432374 21551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:24.432988 21551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:24.434084 21551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:24.434161 21551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.434227 21551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:24.434278 21551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.443081 21551 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:40857
I20260929 18:41:24.443164 21678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:40857 every 8 connection(s)
I20260929 18:41:24.444101 21551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:24.451081 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.446485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:55581 (local address 127.11.118.254:39245)
0929 18:41:24.446653 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:24.446657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:24.447824 (+ 1167us) server_negotiation.cc:407] Connection header received
0929 18:41:24.449177 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:24.449180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:24.449226 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:24.449313 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:24.450586 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:24.450589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:24.450592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:24.450611 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:24.450630 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:24.450637 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:24.450758 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:24.450892 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 18:41:24.450938 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 18:41:24.451749 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21551
I20260929 18:41:24.451869 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:41:24.452104 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.446916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39245 (local address 127.11.118.193:55581)
0929 18:41:24.447656 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:24.447698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:24.448949 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:24.449504 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:24.449511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:24.449603 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:24.450209 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:24.450211 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:24.450359 (+ 148us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:24.450433 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:24.450731 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:24.450740 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:24.450746 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 18:41:24.450940 (+ 194us) client_negotiation.cc:240] Negotiation successful
0929 18:41:24.451206 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":122,"threads_started":1}
I20260929 18:41:24.452179 11739 external_mini_cluster.cc:1428] Setting key 61f05018b84c5133d2c77ee530d55182
I20260929 18:41:24.453526 21679 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39245
I20260929 18:41:24.453785 21679 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:24.454308 21679 heartbeater.cc:511] Master 127.11.118.254:39245 requested a full tablet report, sending...
I20260929 18:41:24.454680 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--builtin_ntp_servers=127.11.118.212:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 18:41:24.456148 21489 ts_manager.cc:194] Registered new tserver with Master: 8f27e805fd534c9ca0099e2fd4f31a2c (127.11.118.193:40857)
W20260929 18:41:24.570217 21682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:24.570602 21682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:24.570691 21682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:24.576542 21682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:24.576634 21682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:24.576732 21682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:24.581893 21682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:24.583151 21682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:24.584581 21682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:24.594624 21688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.594619 21690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.594733 21682 server_base.cc:1034] running on GCE node
W20260929 18:41:24.594619 21687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.595377 21682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:24.595979 21682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:24.597167 21682 hybrid_clock.cc:648] HybridClock initialized: now 1790707284597151 us; error 35 us; skew 500 ppm
I20260929 18:41:24.599349 21682 webserver.cc:460] Webserver started at http://127.11.118.194:33083/ using document root <none> and password file <none>
I20260929 18:41:24.599951 21682 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:24.600016 21682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:24.600262 21682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:24.602469 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "ef84e05d9efe43dba8773dd254fa651b"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "5b92ef0ca0f65bb9d07ad9dcec94aa22"
server_key_iv: "b954594e0d6f500f1453f600ef5d886f"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.603242 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "ef84e05d9efe43dba8773dd254fa651b"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "5b92ef0ca0f65bb9d07ad9dcec94aa22"
server_key_iv: "b954594e0d6f500f1453f600ef5d886f"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.607371 21682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 18:41:24.609921 21696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.611208 21682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:41:24.611320 21682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "ef84e05d9efe43dba8773dd254fa651b"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "5b92ef0ca0f65bb9d07ad9dcec94aa22"
server_key_iv: "b954594e0d6f500f1453f600ef5d886f"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.611413 21682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:24.616024 21682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:24.616603 21682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:24.616748 21682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:24.617286 21682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:24.618181 21682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:24.618232 21682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.618325 21682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:24.618362 21682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.627681 21682 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:39495
I20260929 18:41:24.627707 21809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:39495 every 8 connection(s)
I20260929 18:41:24.628691 21682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:24.631628 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21682
I20260929 18:41:24.631768 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:41:24.632087 11739 external_mini_cluster.cc:1428] Setting key 71b8c5268adc7193fa50f3f6c6be8008
I20260929 18:41:24.634472 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--builtin_ntp_servers=127.11.118.212:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 18:41:24.635864 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.630793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:50939 (local address 127.11.118.254:39245)
0929 18:41:24.630972 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:24.630977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:24.632336 (+ 1359us) server_negotiation.cc:407] Connection header received
0929 18:41:24.633941 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:24.633943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:24.633977 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:24.634053 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:24.635415 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:24.635418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:24.635420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:24.635431 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:24.635451 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:24.635459 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:24.635541 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:24.635712 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 18:41:24.635748 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 18:41:24.637149 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.631326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39245 (local address 127.11.118.194:50939)
0929 18:41:24.632136 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:24.632212 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:24.633684 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:24.634312 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:24.634326 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:24.634463 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:24.635064 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:24.635068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:24.635129 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:24.635224 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:24.635553 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:24.635564 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:24.635573 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 18:41:24.635752 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 18:41:24.636030 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":114,"threads_started":1}
I20260929 18:41:24.638480 21810 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39245
I20260929 18:41:24.638728 21810 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:24.639156 21810 heartbeater.cc:511] Master 127.11.118.254:39245 requested a full tablet report, sending...
I20260929 18:41:24.640267 21489 ts_manager.cc:194] Registered new tserver with Master: ef84e05d9efe43dba8773dd254fa651b (127.11.118.194:39495)
W20260929 18:41:24.749718 21813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:24.749993 21813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:24.750032 21813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:24.754153 21813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:24.754220 21813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:24.754295 21813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:24.759099 21813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:24.760171 21813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:24.761670 21813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:24.771301 21818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.771392 21819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:24.771402 21821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:24.771615 21813 server_base.cc:1034] running on GCE node
I20260929 18:41:24.772035 21813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:24.772725 21813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:24.773950 21813 hybrid_clock.cc:648] HybridClock initialized: now 1790707284773912 us; error 49 us; skew 500 ppm
I20260929 18:41:24.776088 21813 webserver.cc:460] Webserver started at http://127.11.118.195:42355/ using document root <none> and password file <none>
I20260929 18:41:24.776714 21813 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:24.776815 21813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:24.777060 21813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:24.779276 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "c621a9b3903e438e834f144284699842"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "c95a96283d27dd2c3720f5e3f2c6db16"
server_key_iv: "c8315c3f5985d925d3634827383d90c8"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.779866 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "c621a9b3903e438e834f144284699842"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "c95a96283d27dd2c3720f5e3f2c6db16"
server_key_iv: "c8315c3f5985d925d3634827383d90c8"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.783667 21813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:41:24.786403 21827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.787667 21813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:41:24.787832 21813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "c621a9b3903e438e834f144284699842"
format_stamp: "Formatted at 2026-09-29 18:41:24 on dist-test-slave-cgv7"
server_key: "c95a96283d27dd2c3720f5e3f2c6db16"
server_key_iv: "c8315c3f5985d925d3634827383d90c8"
server_key_version: "encryptionkey@0"
I20260929 18:41:24.787962 21813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:24.792956 21813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:24.793576 21813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:24.793766 21813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:24.794392 21813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:24.795348 21813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:24.795418 21813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.795481 21813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:24.795531 21813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:24.804364 21813 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:37307
I20260929 18:41:24.804389 21940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:37307 every 8 connection(s)
I20260929 18:41:24.805357 21813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:24.811749 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21813
I20260929 18:41:24.811865 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:41:24.811862 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.807366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:44187 (local address 127.11.118.254:39245)
0929 18:41:24.807529 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:24.807533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:24.808563 (+ 1030us) server_negotiation.cc:407] Connection header received
0929 18:41:24.809851 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:24.809854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:24.809892 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:24.809973 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:24.811338 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:24.811341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:24.811343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:41:24.811354 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:24.811370 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:41:24.811377 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:24.811492 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:24.811649 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 18:41:24.811700 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 18:41:24.812206 11739 external_mini_cluster.cc:1428] Setting key e370bc02170df7061d0adfc9d8ecf13c
I20260929 18:41:24.812896 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:24.807733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:39245 (local address 127.11.118.195:44187)
0929 18:41:24.808404 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:24.808446 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:24.809661 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:24.810184 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:24.810222 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:24.810402 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:24.811057 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:41:24.811058 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:24.811100 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:41:24.811173 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:24.811491 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:24.811499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:24.811504 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:41:24.811660 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 18:41:24.811943 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260929 18:41:24.814007 21941 heartbeater.cc:347] Connected to a master server at 127.11.118.254:39245
I20260929 18:41:24.814251 21941 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:24.814765 21941 heartbeater.cc:511] Master 127.11.118.254:39245 requested a full tablet report, sending...
I20260929 18:41:24.815855 21491 ts_manager.cc:194] Registered new tserver with Master: c621a9b3903e438e834f144284699842 (127.11.118.195:37307)
I20260929 18:41:24.826612 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 18:41:24.829866 21530 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:41:24.828829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.11.118.254:39245)
0929 18:41:24.828994 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:24.828998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:24.829028 (+ 30us) server_negotiation.cc:407] Connection header received
0929 18:41:24.829163 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:24.829165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:24.829200 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:24.829268 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:24.829732 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55344: BlockingRecv error: recv got EOF from 127.0.0.1:55344 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 18:41:24.829986 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21551
I20260929 18:41:24.835445 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21682
I20260929 18:41:24.840801 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21813
I20260929 18:41:24.846129 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21460
2026-09-29T18:41:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (840 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:41:24 dist-test-slave-cgv7 krb5kdc[21951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:41:24 dist-test-slave-cgv7 krb5kdc[21951](info): set up 2 sockets
Sep 29 18:41:24 dist-test-slave-cgv7 krb5kdc[21951](info): commencing operation
krb5kdc: starting...
W20260929 18:41:25.891415 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:41:25 dist-test-slave-cgv7 krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:41:26.018107 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41919
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:39403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:26.130523 21964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:26.130842 21964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:26.130926 21964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:26.135120 21964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:41:26.135217 21964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:26.135254 21964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:26.135315 21964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:41:26.135375 21964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:41:26.141396 21964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:26.142684 21964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:26.143754 21964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:26.152743 21970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.152727 21969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.152731 21972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:26.152837 21964 server_base.cc:1034] running on GCE node
I20260929 18:41:26.153672 21964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:26.154946 21964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:26.156155 21964 hybrid_clock.cc:648] HybridClock initialized: now 1790707286156103 us; error 60 us; skew 500 ppm
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:26.163098 21964 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:41:26.164150 21964 webserver.cc:460] Webserver started at http://127.11.118.254:35303/ using document root <none> and password file <none>
I20260929 18:41:26.164754 21964 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:26.164821 21964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:26.165069 21964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:26.167405 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "000b4eb6f723445381e6e232a68e8e1b"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "6d75041e2842af5403184c179ec069d2"
server_key_iv: "a29be808107b544367230f2fda50a891"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.168015 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "000b4eb6f723445381e6e232a68e8e1b"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "6d75041e2842af5403184c179ec069d2"
server_key_iv: "a29be808107b544367230f2fda50a891"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.171929 21964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:41:26.174587 21979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.175864 21964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:41:26.176015 21964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "000b4eb6f723445381e6e232a68e8e1b"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "6d75041e2842af5403184c179ec069d2"
server_key_iv: "a29be808107b544367230f2fda50a891"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.176132 21964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:26.190014 21964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:26.193146 21964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:26.193360 21964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:26.200666 21964 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:41919
I20260929 18:41:26.200960 22031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:41919 every 8 connection(s)
I20260929 18:41:26.201705 21964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:41:26.204566 22032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:26.204869 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 21964
I20260929 18:41:26.205006 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:41:26.205307 11739 external_mini_cluster.cc:1428] Setting key 475f2e340268857e2932663db4ea43f8
I20260929 18:41:26.212070 22032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: Bootstrap starting.
I20260929 18:41:26.214660 22032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: Neither blocks nor log segments found. Creating new log.
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:26.215626 22032 log.cc:824] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:26.217903 22032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: No bootstrap required, opened a new log
I20260929 18:41:26.220674 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.206800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.11.118.254:41919)
0929 18:41:26.207221 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.207238 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.207283 (+ 45us) server_negotiation.cc:407] Connection header received
0929 18:41:26.208024 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.208046 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.208458 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.208831 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:26.209831 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.211377 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.212309 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.212811 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.213052 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.215936 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:26.215942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:26.215958 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:26.215990 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:26.217747 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.218334 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.218340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.218345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.218464 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.218754 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.218756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.218757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.219167 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:26.219314 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.219604 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.219926 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260929 18:41:26.222107 22032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } }
I20260929 18:41:26.222405 22032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:26.222462 22032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000b4eb6f723445381e6e232a68e8e1b, State: Initialized, Role: FOLLOWER
I20260929 18:41:26.222899 22032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } }
I20260929 18:41:26.223073 22032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:41:26.223141 22032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:41:26.223253 22032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:26.224181 22032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } }
I20260929 18:41:26.224535 22032 leader_election.cc:302] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 000b4eb6f723445381e6e232a68e8e1b; no voters:
I20260929 18:41:26.224840 22032 leader_election.cc:288] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:41:26.224982 22037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:26.225211 22037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [term 1 LEADER]: Becoming Leader. State: Replica: 000b4eb6f723445381e6e232a68e8e1b, State: Running, Role: LEADER
I20260929 18:41:26.225574 22037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } }
I20260929 18:41:26.226075 22032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:41:26.227375 22038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "000b4eb6f723445381e6e232a68e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } } }
I20260929 18:41:26.227372 22039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 000b4eb6f723445381e6e232a68e8e1b. Latest consensus state: current_term: 1 leader_uuid: "000b4eb6f723445381e6e232a68e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "000b4eb6f723445381e6e232a68e8e1b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41919 } } }
I20260929 18:41:26.227526 22038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:26.227526 22039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b [sys.catalog]: This master's current role is: LEADER
I20260929 18:41:26.228091 22046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:41:26.231350 22046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:41:26.236982 22046 catalog_manager.cc:1347] Generated new cluster ID: 7605d08159834af39d7d46aac760d2ad
I20260929 18:41:26.237072 22046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:41:26.242750 22046 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:41:26.243697 22046 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:41:26.250072 22046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: Generated new TSK 0
I20260929 18:41:26.250952 22046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:41:26.285981 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:39403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:26.401764 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:26.402070 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:26.402117 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:26.406217 22058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:26.406312 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:26.406376 22058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:41:26.411868 22058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:26.412976 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:26.413973 22058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:26.423697 22063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.423713 22066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.423763 22064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:26.424000 22058 server_base.cc:1034] running on GCE node
I20260929 18:41:26.424396 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:26.425065 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:26.426250 22058 hybrid_clock.cc:648] HybridClock initialized: now 1790707286426229 us; error 35 us; skew 500 ppm
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:26.429200 22058 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:41:26.430145 22058 webserver.cc:460] Webserver started at http://127.11.118.193:44599/ using document root <none> and password file <none>
I20260929 18:41:26.430775 22058 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:26.430871 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:26.431131 22058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:26.433239 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "9a3867d31f5b7909b127ddef5256973b"
server_key_iv: "3fca4b084b3c7c465319e71f27f40821"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.433815 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "9a3867d31f5b7909b127ddef5256973b"
server_key_iv: "3fca4b084b3c7c465319e71f27f40821"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.437588 22058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.001s
I20260929 18:41:26.440245 22073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.441433 22058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 18:41:26.441576 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "9a3867d31f5b7909b127ddef5256973b"
server_key_iv: "3fca4b084b3c7c465319e71f27f40821"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.441694 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:26.452486 22058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:26.455477 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:26.455688 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:26.456277 22058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:26.457122 22058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:26.457204 22058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.457300 22058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:26.457353 22058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.466795 22058 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:40627
I20260929 18:41:26.466812 22186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:40627 every 8 connection(s)
I20260929 18:41:26.467757 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:41:26.473066 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 22058
I20260929 18:41:26.473220 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:41:26.473549 11739 external_mini_cluster.cc:1428] Setting key b0124df9357153239b0df7c5787cbd11
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:26.482913 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.469646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39353 (local address 127.11.118.254:41919)
0929 18:41:26.469830 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.469835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.470960 (+ 1125us) server_negotiation.cc:407] Connection header received
0929 18:41:26.471782 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.471784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.471829 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.471914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:26.473166 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.473950 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.475340 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.475745 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.475875 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.478782 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:26.478789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:26.478795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:26.478830 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:26.480618 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.481161 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.481169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.481171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.481275 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.481623 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.481628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.481632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.481884 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:26.481996 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.482575 (+ 579us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.482719 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:41:26.483733 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.470080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41919 (local address 127.11.118.193:39353)
0929 18:41:26.470799 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.470842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.471600 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.472072 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.472079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.472480 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:26.473000 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.473015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.474099 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.474106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.475210 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.475222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.476017 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.476022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.476255 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.476971 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:26.476989 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:26.478595 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:26.480743 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.480748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.480763 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.481045 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.481371 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.481373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.481375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.481517 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.481991 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:26.481996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:26.482243 (+ 247us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.482479 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.482749 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":178,"threads_started":1}
I20260929 18:41:26.485242 22187 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41919
I20260929 18:41:26.485540 22187 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:26.486075 22187 heartbeater.cc:511] Master 127.11.118.254:41919 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:41:26.487852 21996 ts_manager.cc:194] Registered new tserver with Master: 66ef07c8b7bc4e11888c3ac2c2d20b3d (127.11.118.193:40627)
I20260929 18:41:26.489148 21996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:39353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:41:26.503714 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:39403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:26.619681 22192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:26.620044 22192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:26.620106 22192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:26.624487 22192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:26.624570 22192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:26.624634 22192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:41:26.630463 22192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:26.631572 22192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:26.632675 22192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:26.642817 22200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.642855 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:26.642952 22192 server_base.cc:1034] running on GCE node
W20260929 18:41:26.642818 22198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:26.643656 22192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:26.644348 22192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:26.645545 22192 hybrid_clock.cc:648] HybridClock initialized: now 1790707286645525 us; error 32 us; skew 500 ppm
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:26.648517 22192 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:41:26.649396 22192 webserver.cc:460] Webserver started at http://127.11.118.194:44215/ using document root <none> and password file <none>
I20260929 18:41:26.650017 22192 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:26.650084 22192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:26.650254 22192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:26.652426 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "b6692602187041cee4df13c977863190"
server_key_iv: "f91afd7e915fcd716518993472a80cab"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.652961 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "b6692602187041cee4df13c977863190"
server_key_iv: "f91afd7e915fcd716518993472a80cab"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.656806 22192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260929 18:41:26.659396 22207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.660692 22192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:41:26.660789 22192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "b6692602187041cee4df13c977863190"
server_key_iv: "f91afd7e915fcd716518993472a80cab"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.660879 22192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:26.689007 22192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:26.692263 22192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:26.692444 22192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:26.692986 22192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:26.693759 22192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:26.693807 22192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.693848 22192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:26.693863 22192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.703292 22192 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:39067
I20260929 18:41:26.703326 22320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:39067 every 8 connection(s)
I20260929 18:41:26.704241 22192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:41:26.710975 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 22192
I20260929 18:41:26.711102 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:41:26.711396 11739 external_mini_cluster.cc:1428] Setting key 9c430c28325a6be4cef539e35dac1bba
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:26.719394 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.706202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:37293 (local address 127.11.118.254:41919)
0929 18:41:26.706432 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.706435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.707382 (+ 947us) server_negotiation.cc:407] Connection header received
0929 18:41:26.708727 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.708730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.708775 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.708856 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:26.710067 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.710750 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.712243 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.712480 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.712598 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.715547 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:26.715550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:26.715552 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:26.715577 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:26.717103 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.717654 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.717658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.717660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.717744 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.718094 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.718099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.718100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.718363 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:26.718472 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.719081 (+ 609us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.719206 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260929 18:41:26.720223 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.706691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41919 (local address 127.11.118.194:37293)
0929 18:41:26.707170 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.707222 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.708532 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.709046 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.709058 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.709454 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:26.709934 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.709948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.710883 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.710887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.712103 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.712114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.712634 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.712638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.712853 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.713532 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:26.713558 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:26.715354 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:26.717229 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.717235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.717253 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.717543 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.717841 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.717845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.717847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.717996 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.718472 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:26.718481 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:26.718830 (+ 349us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.719024 (+ 194us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.719365 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:41:26.721426 22321 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41919
I20260929 18:41:26.721685 22321 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:26.722185 22321 heartbeater.cc:511] Master 127.11.118.254:41919 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:41:26.723344 21996 ts_manager.cc:194] Registered new tserver with Master: 49bcc2c33e564ad0bdc3cfe63f2d6532 (127.11.118.194:39067)
I20260929 18:41:26.723949 21996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:37293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:41:26.741652 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:39403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:41:26.853982 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:41:26.854256 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:41:26.854385 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:41:26.858431 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:41:26.858490 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:41:26.858593 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:41:26.864005 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:41:26.865151 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:41:26.866173 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:41:26.875710 22332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.875771 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:41:26.875721 22334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:41:26.875931 22326 server_base.cc:1034] running on GCE node
I20260929 18:41:26.876508 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:41:26.877118 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:41:26.878453 22326 hybrid_clock.cc:648] HybridClock initialized: now 1790707286878420 us; error 43 us; skew 500 ppm
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:41:26.881448 22326 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:41:26.882478 22326 webserver.cc:460] Webserver started at http://127.11.118.195:35161/ using document root <none> and password file <none>
I20260929 18:41:26.883069 22326 fs_manager.cc:359] Metadata directory not provided
I20260929 18:41:26.883168 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:41:26.883421 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:41:26.885527 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "be19705bc9de430fa2459d5224c58da8"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "e226f9b797f8454184709c2934573bce"
server_key_iv: "73d71bca52704f0c501ef80f2571d676"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.886116 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "be19705bc9de430fa2459d5224c58da8"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "e226f9b797f8454184709c2934573bce"
server_key_iv: "73d71bca52704f0c501ef80f2571d676"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.890029 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260929 18:41:26.892719 22341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.893955 22326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:41:26.894096 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "be19705bc9de430fa2459d5224c58da8"
format_stamp: "Formatted at 2026-09-29 18:41:26 on dist-test-slave-cgv7"
server_key: "e226f9b797f8454184709c2934573bce"
server_key_iv: "73d71bca52704f0c501ef80f2571d676"
server_key_version: "encryptionkey@0"
I20260929 18:41:26.894217 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:41:26.903666 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:41:26.906800 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:41:26.907027 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:41:26.907648 22326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:41:26.908582 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:41:26.908655 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.908728 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:41:26.908779 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:41:26.918798 22326 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:38853
I20260929 18:41:26.918861 22454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:38853 every 8 connection(s)
I20260929 18:41:26.919812 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:41:26.928552 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 22326
I20260929 18:41:26.928699 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:41:26.929014 11739 external_mini_cluster.cc:1428] Setting key c80cd39dbdd26f6bae5ab6031e7d11e4
Sep 29 18:41:26 dist-test-slave-cgv7 krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:41:26.934967 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.921808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:54513 (local address 127.11.118.254:41919)
0929 18:41:26.921964 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.921968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.922905 (+ 937us) server_negotiation.cc:407] Connection header received
0929 18:41:26.924303 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.924306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.924356 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.924434 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:26.925663 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.926322 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.927749 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.928137 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.928272 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.931265 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:26.931272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:26.931277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:26.931332 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:26.932804 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.933416 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.933419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.933421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.933511 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.933870 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.933872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.933874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.934056 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:26.934133 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.934730 (+ 597us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.934848 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 18:41:26.935930 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.922212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:41919 (local address 127.11.118.195:54513)
0929 18:41:26.922726 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.922773 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.924048 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.924609 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.924616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.925000 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:41:26.925482 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.925495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.926527 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.926529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.927621 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.927631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.928285 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.928288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.928438 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.929146 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:41:26.929162 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:41:26.931083 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:41:26.932987 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.932992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.933008 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.933290 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.933612 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:41:26.933614 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:41:26.933615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:41:26.933777 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:41:26.934180 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:41:26.934186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:41:26.934484 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.934664 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.934953 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":119,"threads_started":1}
I20260929 18:41:26.937005 22455 heartbeater.cc:347] Connected to a master server at 127.11.118.254:41919
I20260929 18:41:26.937268 22455 heartbeater.cc:464] Registering TS with master...
I20260929 18:41:26.937768 22455 heartbeater.cc:511] Master 127.11.118.254:41919 requested a full tablet report, sending...
I20260929 18:41:26.938856 21996 ts_manager.cc:194] Registered new tserver with Master: be19705bc9de430fa2459d5224c58da8 (127.11.118.195:38853)
I20260929 18:41:26.939391 21996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:54513
I20260929 18:41:26.943132 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:41:26.953230 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.945160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.11.118.254:41919)
0929 18:41:26.945366 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.945371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.945553 (+ 182us) server_negotiation.cc:407] Connection header received
0929 18:41:26.945759 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.945762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.945803 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.945894 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:41:26.946730 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.947263 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.948119 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.948344 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.948454 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.949935 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:41:26.949943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:41:26.949946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:41:26.949970 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:41:26.951394 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.951859 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.951864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.951866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.951951 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:41:26.952245 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:41:26.952250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:41:26.952252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:41:26.952522 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:41:26.952669 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.952925 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.953063 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:41:26.956324 21996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:41:26.959347 21996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:41:26.974553 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.968054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:40627 (local address 127.0.0.1:37028)
0929 18:41:26.968418 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.968466 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.968981 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.969667 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.969672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.969698 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:26.969850 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.969858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971338 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.971341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.972919 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.972927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.973941 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.973945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.974066 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.974124 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.974238 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.974394 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":264,"thread_start_us":133,"threads_started":1}
I20260929 18:41:26.974694 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.968798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:38853 (local address 127.0.0.1:49670)
0929 18:41:26.969423 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.969435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.969514 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.970376 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.970379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.970397 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:26.970496 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.970501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971731 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.971734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.973230 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.973241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.974161 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.974165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.974314 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.974340 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.974409 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.974483 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":144,"thread_start_us":85,"threads_started":1}
I20260929 18:41:26.974709 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.968339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:39067 (local address 127.0.0.1:51028)
0929 18:41:26.968847 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:26.968863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:26.968980 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:26.970192 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:26.970197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:26.970219 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:26.970366 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.970373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971631 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.971637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.973230 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:26.973241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.974338 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.974341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:26.974437 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.974454 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 18:41:26.974496 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 18:41:26.974547 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":192,"thread_start_us":80,"threads_started":1}
I20260929 18:41:26.975174 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.968200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.11.118.193:40627)
0929 18:41:26.969156 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.969171 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.969194 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:41:26.969270 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.969275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.969500 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.969658 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:26.969976 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971186 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.973043 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.973806 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.973933 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.974067 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.974920 (+ 853us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.975034 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":85,"threads_started":1}
I20260929 18:41:26.975179 22471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.969342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.11.118.195:38853)
0929 18:41:26.969849 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.969860 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.969884 (+ 24us) server_negotiation.cc:407] Connection header received
0929 18:41:26.969960 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.969965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.970130 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.970620 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:26.970734 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971554 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.973399 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.974006 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.974163 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.974329 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.974438 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.974530 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":137,"threads_started":1}
I20260929 18:41:26.975436 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:26.968688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.11.118.194:39067)
0929 18:41:26.969749 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:26.969760 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:26.969781 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:41:26.969860 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:26.969864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:26.970054 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:26.970202 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:26.970485 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.971498 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.973360 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:26.974173 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:26.974812 (+ 639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:26.975120 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:26.975171 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 18:41:26.975284 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":966,"thread_start_us":138,"threads_started":1}
I20260929 18:41:26.978382 22255 tablet_service.cc:1467] Processing CreateTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 (DEFAULT_TABLE table=test-table [id=7abd7a73fa9d458892a0fc3453d07ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:26.978382 22389 tablet_service.cc:1467] Processing CreateTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 (DEFAULT_TABLE table=test-table [id=7abd7a73fa9d458892a0fc3453d07ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:26.978379 22121 tablet_service.cc:1467] Processing CreateTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 (DEFAULT_TABLE table=test-table [id=7abd7a73fa9d458892a0fc3453d07ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:41:26.979444 22255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ccc6cbb2bc46c082448c20612a9942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:26.979455 22121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ccc6cbb2bc46c082448c20612a9942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:26.979462 22389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ccc6cbb2bc46c082448c20612a9942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:41:26.985999 22472 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Bootstrap starting.
I20260929 18:41:26.987457 22474 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Bootstrap starting.
I20260929 18:41:26.988662 22472 tablet_bootstrap.cc:654] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:26.989290 22474 tablet_bootstrap.cc:654] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:26.989521 22472 log.cc:824] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:26.990161 22474 log.cc:824] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:26.990531 22473 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Bootstrap starting.
I20260929 18:41:26.992105 22474 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: No bootstrap required, opened a new log
I20260929 18:41:26.992355 22474 ts_tablet_manager.cc:1397] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 18:41:26.992705 22473 tablet_bootstrap.cc:654] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Neither blocks nor log segments found. Creating new log.
I20260929 18:41:26.993388 22473 log.cc:824] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Log is configured to *not* fsync() on all Append() calls
I20260929 18:41:26.995091 22472 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: No bootstrap required, opened a new log
I20260929 18:41:26.995308 22472 ts_tablet_manager.cc:1397] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260929 18:41:26.995512 22473 tablet_bootstrap.cc:492] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: No bootstrap required, opened a new log
I20260929 18:41:26.995726 22473 ts_tablet_manager.cc:1397] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:41:26.996553 22474 raft_consensus.cc:348] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:26.996855 22474 raft_consensus.cc:374] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:26.996930 22474 raft_consensus.cc:729] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ef07c8b7bc4e11888c3ac2c2d20b3d, State: Initialized, Role: FOLLOWER
I20260929 18:41:26.997455 22474 consensus_queue.cc:260] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:26.998154 22187 heartbeater.cc:503] Master 127.11.118.254:41919 was elected leader, sending a full tablet report...
I20260929 18:41:26.999686 22474 ts_tablet_manager.cc:1428] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 18:41:26.999500 22472 raft_consensus.cc:348] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:26.999594 22473 raft_consensus.cc:348] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:26.999819 22473 raft_consensus.cc:374] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:26.999841 22472 raft_consensus.cc:374] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:41:26.999899 22472 raft_consensus.cc:729] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49bcc2c33e564ad0bdc3cfe63f2d6532, State: Initialized, Role: FOLLOWER
I20260929 18:41:26.999914 22473 raft_consensus.cc:729] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be19705bc9de430fa2459d5224c58da8, State: Initialized, Role: FOLLOWER
I20260929 18:41:27.000378 22473 consensus_queue.cc:260] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:27.000378 22472 consensus_queue.cc:260] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:27.001533 22473 ts_tablet_manager.cc:1428] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:41:27.001525 22472 ts_tablet_manager.cc:1428] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:41:27.001652 22455 heartbeater.cc:503] Master 127.11.118.254:41919 was elected leader, sending a full tablet report...
I20260929 18:41:27.001828 22321 heartbeater.cc:503] Master 127.11.118.254:41919 was elected leader, sending a full tablet report...
I20260929 18:41:27.168069 22480 raft_consensus.cc:482] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:41:27.168323 22480 raft_consensus.cc:504] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:27.169544 22480 leader_election.cc:288] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ef07c8b7bc4e11888c3ac2c2d20b3d (127.11.118.193:40627), 49bcc2c33e564ad0bdc3cfe63f2d6532 (127.11.118.194:39067)
W20260929 18:41:27.170898 22456 tablet.cc:2369] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:41:27.173780 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.169977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:37593 (local address 127.11.118.194:39067)
0929 18:41:27.170106 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:27.170110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:27.170455 (+ 345us) server_negotiation.cc:407] Connection header received
0929 18:41:27.170592 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:27.170595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:27.170647 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:27.170708 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:27.170970 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.171686 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.172807 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.173185 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.173313 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:27.173370 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:27.173576 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 18:41:27.173649 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:41:27.173780 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.169990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:49811 (local address 127.11.118.193:40627)
0929 18:41:27.170123 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:27.170130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:27.170147 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:41:27.170335 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:27.170339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:27.170389 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:27.170482 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:41:27.171158 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.171651 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.172766 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.173180 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.173320 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:27.173377 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:27.173575 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 18:41:27.173656 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:41:27.173780 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.169859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:40627 (local address 127.11.118.195:49811)
0929 18:41:27.170075 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:27.170090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:27.170208 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:27.170506 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:27.170512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:27.170531 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:27.170656 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:27.170660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.171764 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.171771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:27.172631 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:27.172640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.173307 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.173311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:27.173449 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:27.173472 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:41:27.173559 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 18:41:27.173648 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":42}
I20260929 18:41:27.173792 22481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.169859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:39067 (local address 127.11.118.195:37593)
0929 18:41:27.170341 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:27.170361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 18:41:27.170482 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:41:27.170745 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:41:27.170747 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:41:27.170766 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:41:27.170867 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:27.170871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.171840 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.171843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:27.172660 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:41:27.172668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.173308 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.173311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:41:27.173447 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:27.173469 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:41:27.173559 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 18:41:27.173643 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":84,"thread_start_us":108,"threads_started":1}
I20260929 18:41:27.174544 22141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ccc6cbb2bc46c082448c20612a9942" candidate_uuid: "be19705bc9de430fa2459d5224c58da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" is_pre_election: true
I20260929 18:41:27.174539 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ccc6cbb2bc46c082448c20612a9942" candidate_uuid: "be19705bc9de430fa2459d5224c58da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" is_pre_election: true
I20260929 18:41:27.174867 22141 raft_consensus.cc:2454] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be19705bc9de430fa2459d5224c58da8 in term 0.
I20260929 18:41:27.174877 22275 raft_consensus.cc:2454] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be19705bc9de430fa2459d5224c58da8 in term 0.
I20260929 18:41:27.175386 22343 leader_election.cc:302] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49bcc2c33e564ad0bdc3cfe63f2d6532, be19705bc9de430fa2459d5224c58da8; no voters:
I20260929 18:41:27.175719 22480 raft_consensus.cc:2781] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:41:27.175812 22480 raft_consensus.cc:482] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:41:27.175871 22480 raft_consensus.cc:3037] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:27.176900 22480 raft_consensus.cc:504] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:27.177302 22480 leader_election.cc:288] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [CANDIDATE]: Term 1 election: Requested vote from peers 66ef07c8b7bc4e11888c3ac2c2d20b3d (127.11.118.193:40627), 49bcc2c33e564ad0bdc3cfe63f2d6532 (127.11.118.194:39067)
I20260929 18:41:27.177645 22141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ccc6cbb2bc46c082448c20612a9942" candidate_uuid: "be19705bc9de430fa2459d5224c58da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d"
I20260929 18:41:27.177758 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ccc6cbb2bc46c082448c20612a9942" candidate_uuid: "be19705bc9de430fa2459d5224c58da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532"
I20260929 18:41:27.177824 22141 raft_consensus.cc:3037] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:27.177942 22275 raft_consensus.cc:3037] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:41:27.178992 22141 raft_consensus.cc:2454] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be19705bc9de430fa2459d5224c58da8 in term 1.
I20260929 18:41:27.179153 22275 raft_consensus.cc:2454] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be19705bc9de430fa2459d5224c58da8 in term 1.
I20260929 18:41:27.179322 22342 leader_election.cc:302] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ef07c8b7bc4e11888c3ac2c2d20b3d, be19705bc9de430fa2459d5224c58da8; no voters:
I20260929 18:41:27.179509 22480 raft_consensus.cc:2781] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:41:27.179826 22480 raft_consensus.cc:686] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 1 LEADER]: Becoming Leader. State: Replica: be19705bc9de430fa2459d5224c58da8, State: Running, Role: LEADER
I20260929 18:41:27.180194 22480 consensus_queue.cc:237] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } }
I20260929 18:41:27.183621 21993 catalog_manager.cc:5495] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 reported cstate change: term changed from 0 to 1, leader changed from <none> to be19705bc9de430fa2459d5224c58da8 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "be19705bc9de430fa2459d5224c58da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be19705bc9de430fa2459d5224c58da8" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 38853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 } health_report { overall_health: UNKNOWN } } }
W20260929 18:41:27.205474 22322 tablet.cc:2369] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:41:27.217634 22471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.214229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.11.118.195:38853)
0929 18:41:27.214391 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 18:41:27.214395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:41:27.214409 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:41:27.214522 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:41:27.214524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:41:27.214579 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:41:27.214666 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:41:27.214892 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.215382 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.216245 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:41:27.216465 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:41:27.216547 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:41:27.216833 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:41:27.217244 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:41:27.217338 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 18:41:27.217470 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 18:41:27.217524 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260929 18:41:27.219087 22188 tablet.cc:2369] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:41:27.223482 22141 raft_consensus.cc:1261] T a2ccc6cbb2bc46c082448c20612a9942 P 66ef07c8b7bc4e11888c3ac2c2d20b3d [term 1 FOLLOWER]: Refusing update from remote peer be19705bc9de430fa2459d5224c58da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:27.223481 22275 raft_consensus.cc:1261] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 1 FOLLOWER]: Refusing update from remote peer be19705bc9de430fa2459d5224c58da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:41:27.224165 22480 consensus_queue.cc:1035] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ef07c8b7bc4e11888c3ac2c2d20b3d" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 40627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:27.224321 22482 consensus_queue.cc:1035] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49bcc2c33e564ad0bdc3cfe63f2d6532" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:41:27.232421 22487 mvcc.cc:205] Tried to move back new op lower bound from 7334737048460447744 to 7334737048293339136. Current Snapshot: MvccSnapshot[applied={T|T < 7334737048460447744}]
I20260929 18:41:27.240751 21995 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:27.240917 21995 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:41:27.243544 21995 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 000b4eb6f723445381e6e232a68e8e1b: Sending DeleteTablet for 3 replicas of tablet a2ccc6cbb2bc46c082448c20612a9942
I20260929 18:41:27.244484 22255 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:27 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:51028
I20260929 18:41:27.244485 22121 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:27 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:37028
I20260929 18:41:27.244552 22389 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ccc6cbb2bc46c082448c20612a9942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:41:27 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49670
I20260929 18:41:27.245244 22496 tablet_replica.cc:330] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: stopping tablet replica
I20260929 18:41:27.245637 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 22058
I20260929 18:41:27.245533 22495 tablet_replica.cc:330] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: stopping tablet replica
I20260929 18:41:27.247884 22495 raft_consensus.cc:2229] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:41:27.248179 22495 raft_consensus.cc:2258] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:27.248621 22496 raft_consensus.cc:2229] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:41:27.249086 22496 raft_consensus.cc:2258] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:41:27.249711 22495 ts_tablet_manager.cc:1905] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:27.250473 22496 ts_tablet_manager.cc:1905] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:41:27.252823 22495 ts_tablet_manager.cc:1918] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:27.253060 22495 log.cc:1196] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/a2ccc6cbb2bc46c082448c20612a9942
I20260929 18:41:27.253474 22495 ts_tablet_manager.cc:1939] T a2ccc6cbb2bc46c082448c20612a9942 P 49bcc2c33e564ad0bdc3cfe63f2d6532: Deleting consensus metadata
I20260929 18:41:27.253984 22496 ts_tablet_manager.cc:1918] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:41:27.254174 22496 log.cc:1196] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/a2ccc6cbb2bc46c082448c20612a9942
I20260929 18:41:27.254590 22496 ts_tablet_manager.cc:1939] T a2ccc6cbb2bc46c082448c20612a9942 P be19705bc9de430fa2459d5224c58da8: Deleting consensus metadata
I20260929 18:41:27.255080 21981 catalog_manager.cc:4841] TS 49bcc2c33e564ad0bdc3cfe63f2d6532 (127.11.118.194:39067): tablet a2ccc6cbb2bc46c082448c20612a9942 (table test-table [id=7abd7a73fa9d458892a0fc3453d07ad9]) successfully deleted
I20260929 18:41:27.255846 21980 catalog_manager.cc:4841] TS be19705bc9de430fa2459d5224c58da8 (127.11.118.195:38853): tablet a2ccc6cbb2bc46c082448c20612a9942 (table test-table [id=7abd7a73fa9d458892a0fc3453d07ad9]) successfully deleted
I20260929 18:41:27.256107 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 22192
W20260929 18:41:27.256116 21980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:40627 (error 108)
I20260929 18:41:27.259545 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:41:27.258415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:40627 (local address 127.0.0.1:37040)
0929 18:41:27.259351 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0929 18:41:27.259437 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:40627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":802}
W20260929 18:41:27.259801 21980 catalog_manager.cc:4584] TS 66ef07c8b7bc4e11888c3ac2c2d20b3d (127.11.118.193:40627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2ccc6cbb2bc46c082448c20612a9942: Network error: Client connection negotiation failed: client connection to 127.11.118.193:40627: connect: Connection refused (error 111)
I20260929 18:41:27.262602 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 22326
I20260929 18:41:27.268577 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 21964
2026-09-29T18:41:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2422 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:41:27 dist-test-slave-cgv7 krb5kdc[22501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:41:27 dist-test-slave-cgv7 krb5kdc[22501](info): set up 2 sockets
Sep 29 18:41:27 dist-test-slave-cgv7 krb5kdc[22501](info): commencing operation
krb5kdc: starting...
W20260929 18:41:28.319363 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:41:28 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:41:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangeradmin_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangeradmin_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangerlookup_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangerlookup_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
I20260929 18:41:28.498993 11739 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqumnYE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/postgres -l logfile start
2026-09-29 18:41:31.212 UTC [22531] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 18:41:31.212 UTC [22531] LOG: listening on IPv4 address "127.11.118.212", port 54191
2026-09-29 18:41:31.217 UTC [22531] LOG: listening on Unix socket "/tmp/.s.PGSQL.54191"
2026-09-29 18:41:31.251 UTC [22534] LOG: database system was shut down at 2026-09-29 18:41:29 UTC
2026-09-29 18:41:31.255 UTC [22531] LOG: database system is ready to accept connections
I20260929 18:41:32.207064 11739 mini_postgres.cc:96] Postgres bound to 54191
2026-09-29 18:41:32.213 UTC [22542] FATAL: database "slave" does not exist
127.11.118.212:54191 - accepting connections
I20260929 18:41:32.214448 11739 mini_ranger.cc:161] Starting Ranger...
I20260929 18:41:32.229708 11739 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 18:41:32.779850 11739 mini_ranger.cc:87] Created ranger Postgres database
I20260929 18:41:32.779961 11739 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin
2026-09-29 18:41:33,037 [I] DB FLAVOR :POSTGRES
2026-09-29 18:41:33,037 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 18:41:33,037 [I] Checking connection..
2026-09-29 18:41:33,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 18:41:33,289 [I] Checking connection passed.
2026-09-29 18:41:33,289 [I] --------- Verifying version history table ---------
2026-09-29 18:41:33,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:41:33,536 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 18:41:33,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:41:33,795 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 18:41:33,795 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 18:41:33,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 18:41:34.055 UTC [22606] WARNING: there is no transaction in progress
2026-09-29 18:41:34,061 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 18:41:34,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:41:34,307 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 18:41:34,308 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 18:41:34,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 18:41:34,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 18:41:34,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cgv7.c.gcp-upstream.internal','N') ;"
2026-09-29 18:41:35,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 18:41:35,322 [I] Table x_portal_user does not exist in database ranger
2026-09-29 18:41:35,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 18:41:35,557 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 18:41:35,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 18:41:35,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 18:41:36,025 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 18:41:36,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 18:41:37.025 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.043 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.061 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.077 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.095 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.113 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.168 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.175 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.189 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.200 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.210 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.225 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.236 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.243 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.251 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37.258 UTC [22714] WARNING: there is no transaction in progress
2026-09-29 18:41:37,993 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 18:41:37,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 18:41:38,249 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 18:41:38,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 18:41:38,514 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 18:41:38,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 18:41:38,748 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 18:41:38,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 18:41:38,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 18:41:39,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cgv7.c.gcp-upstream.internal';"
2026-09-29 18:41:39,470 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 18:41:39,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 18:41:39,707 [I] DB_PATCHES have already been applied
I20260929 18:41:39.712663 11739 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqumnYE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 18:41:39.712733 11739 mini_ranger.cc:193] Using host: 127.11.118.212
I20260929 18:41:39.716135 11739 mini_ranger.cc:227] Ranger admin URL: http://127.11.118.212:39947
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 6:41:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 18:41:40.842062 11739 mini_ranger.cc:160] Time spent starting Ranger: real 8.628s user 0.001s sys 0.009s
Sep 29, 2026 6:42:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 6:42:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 6:42:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 6:42:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 6:42:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 18:42:03.117805 11739 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangerkms_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/rangerkms_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.118.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.118.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 18:42:03 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707323, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 18:42:03.205096 11739 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 18:42:03.214807 11739 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 18:42:04.199697 11739 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 18:42:04.199844 11739 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms
I20260929 18:42:04.199956 11739 mini_ranger_kms.cc:227] Using postgres at 127.11.118.212:54191
2026-09-29 18:42:04,378 [I] DB FLAVOR :POSTGRES
2026-09-29 18:42:04,378 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 18:42:04,378 [I] Checking connection
2026-09-29 18:42:04,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 18:42:04,640 [I] connection success
2026-09-29 18:42:04,640 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 18:42:04,640 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 18:42:04,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 18:42:04,897 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 18:42:04,898 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 18:42:04,898 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 18:42:04,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:54191/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 18:42:05,170 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 18:42:05.615429 11739 mini_ranger_kms.cc:310] Created kms service
I20260929 18:42:05.830987 11739 mini_ranger_kms.cc:326] Created kudu user
I20260929 18:42:05.871892 11739 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 18:42:05 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707323, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:06.286561 11739 mini_ranger_kms.cc:384] Added ranger policy
I20260929 18:42:06.287035 11739 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 18:42:06.287082 11739 mini_ranger_kms.cc:240] Using host: 127.11.118.212
I20260929 18:42:06.291075 11739 mini_ranger_kms.cc:276] Ranger KMS PID: 22894
I20260929 18:42:06.291239 11739 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.118.212:34231
Sep 29, 2026 6:42:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:42:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:42:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 6:42:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 6:42:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 6:42:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 18:42:08.534576 11739 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.329s user 0.023s sys 0.139s
I20260929 18:42:08.534711 11739 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 18:42:08.534812 11739 mini_ranger_kms.cc:401] 127.11.118.212:34231/kms/v1/keys
Sep 29 18:42:08 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790707328, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 18:42:09 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707328, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
Sep 29 18:42:09 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707328, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
Sep 29 18:42:12 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:42:12.070057 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33819
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:33819
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:34231/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:42:12.189985 22945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:42:12.190327 22945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:42:12.190407 22945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:42:12.194581 22945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:42:12.194658 22945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:42:12.194679 22945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:42:12.194715 22945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:42:12.194741 22945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:42:12.200556 22945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.118.212:34231/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:33819
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:33819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:42:12.201687 22945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:42:12.202818 22945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:42:12.212423 22950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:12.212522 22945 server_base.cc:1034] running on GCE node
W20260929 18:42:12.212441 22951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:12.212757 22953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:12.213352 22945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:42:12.214562 22945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:42:12.215783 22945 hybrid_clock.cc:648] HybridClock initialized: now 1790707332215761 us; error 41 us; skew 500 ppm
Sep 29 18:42:12 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:42:12.223006 22945 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:42:12.224327 22945 webserver.cc:460] Webserver started at http://127.11.118.254:44397/ using document root <none> and password file <none>
I20260929 18:42:12.225246 22945 fs_manager.cc:359] Metadata directory not provided
I20260929 18:42:12.225342 22945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:42:12.225598 22945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:42:12 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:12.296885 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "c6721aba5de0402f92ea9f221f1ca68d"
format_stamp: "Formatted at 2026-09-29 18:42:12 on dist-test-slave-cgv7"
server_key: "d17bf3858baa3eca1f69f5488335b783"
server_key_iv: "cc2514293ee7c27819a794a8cb2dc5bb"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:12.297662 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "c6721aba5de0402f92ea9f221f1ca68d"
format_stamp: "Formatted at 2026-09-29 18:42:12 on dist-test-slave-cgv7"
server_key: "d17bf3858baa3eca1f69f5488335b783"
server_key_iv: "cc2514293ee7c27819a794a8cb2dc5bb"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:12.302621 22945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 18:42:12.317847 22960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:12.319427 22945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:42:12.319600 22945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "c6721aba5de0402f92ea9f221f1ca68d"
format_stamp: "Formatted at 2026-09-29 18:42:12 on dist-test-slave-cgv7"
server_key: "d17bf3858baa3eca1f69f5488335b783"
server_key_iv: "cc2514293ee7c27819a794a8cb2dc5bb"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:12.319754 22945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:42:12.331249 22945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:42:12.334331 22945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:42:12.334547 22945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:42:12.342612 22945 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:33819
I20260929 18:42:12.342650 23012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:33819 every 8 connection(s)
I20260929 18:42:12.343647 22945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:42:12.346722 23013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:42:12.348285 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 22945
I20260929 18:42:12.348413 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:42:12.348708 11739 external_mini_cluster.cc:1428] Setting key fb51d9afa18014e03543df62a91f9da9
I20260929 18:42:12.353624 23013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: Bootstrap starting.
I20260929 18:42:12.356280 23013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: Neither blocks nor log segments found. Creating new log.
I20260929 18:42:12.357635 23013 log.cc:824] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: Log is configured to *not* fsync() on all Append() calls
Sep 29 18:42:12 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:42:12.360067 23013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: No bootstrap required, opened a new log
I20260929 18:42:12.363590 23013 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } }
I20260929 18:42:12.363940 23013 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:42:12.364017 23013 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6721aba5de0402f92ea9f221f1ca68d, State: Initialized, Role: FOLLOWER
I20260929 18:42:12.364542 23013 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } }
I20260929 18:42:12.364701 23013 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:42:12.364816 23013 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:42:12.364925 23013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:42:12.366200 23013 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } }
I20260929 18:42:12.366581 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:12.350308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.11.118.254:33819)
0929 18:42:12.350763 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:12.350785 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:12.350831 (+ 46us) server_negotiation.cc:407] Connection header received
0929 18:42:12.351741 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:12.351767 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:12.352213 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:12.352597 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:42:12.353467 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:12.354898 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:12.356117 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:12.356824 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:12.356996 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:12.360109 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:42:12.360116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:42:12.360135 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:42:12.360167 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:42:12.362881 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:12.363441 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:12.363447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:12.363453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:12.363587 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:12.363908 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:12.363911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:12.363913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:12.364422 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:42:12.364608 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:12.365295 (+ 687us) server_negotiation.cc:299] Negotiation successful
0929 18:42:12.365566 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":117,"threads_started":1}
I20260929 18:42:12.366583 23013 leader_election.cc:302] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6721aba5de0402f92ea9f221f1ca68d; no voters:
I20260929 18:42:12.366936 23013 leader_election.cc:288] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:42:12.367010 23018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:42:12.367228 23018 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [term 1 LEADER]: Becoming Leader. State: Replica: c6721aba5de0402f92ea9f221f1ca68d, State: Running, Role: LEADER
I20260929 18:42:12.367539 23018 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } }
I20260929 18:42:12.368323 23013 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:42:12.368769 23018 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6721aba5de0402f92ea9f221f1ca68d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } } }
I20260929 18:42:12.368896 23018 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [sys.catalog]: This master's current role is: LEADER
I20260929 18:42:12.369128 23018 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6721aba5de0402f92ea9f221f1ca68d. Latest consensus state: current_term: 1 leader_uuid: "c6721aba5de0402f92ea9f221f1ca68d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6721aba5de0402f92ea9f221f1ca68d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 33819 } } }
I20260929 18:42:12.369298 23018 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d [sys.catalog]: This master's current role is: LEADER
I20260929 18:42:12.369620 23022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:42:12.372875 23022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:42:12.373338 23013 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 18:42:12.388780 23022 catalog_manager.cc:1347] Generated new cluster ID: 19f559f1e4f84bf98ebeb17c9e222c6e
I20260929 18:42:12.388866 23022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:42:12.406154 23022 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:42:12.408233 23022 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:42:12.420037 23022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: Generated new TSK 0
I20260929 18:42:12.421133 23022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:42:13.185367 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:34231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:42:13.311764 23058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:42:13.312155 23058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:42:13.312263 23058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:42:13.317298 23058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:42:13.317390 23058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:42:13.317544 23058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:42:13.325088 23058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.118.212:34231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:42:13.326436 23058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:42:13.327529 23058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:42:13.338551 23066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:13.338773 23068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:13.338551 23065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:13.339154 23058 server_base.cc:1034] running on GCE node
I20260929 18:42:13.339570 23058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:42:13.340283 23058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:42:13.341470 23058 hybrid_clock.cc:648] HybridClock initialized: now 1790707333341436 us; error 47 us; skew 500 ppm
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:42:13.344429 23058 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:42:13.345778 23058 webserver.cc:460] Webserver started at http://127.11.118.193:40453/ using document root <none> and password file <none>
I20260929 18:42:13.346431 23058 fs_manager.cc:359] Metadata directory not provided
I20260929 18:42:13.346526 23058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:42:13.346796 23058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:13.363435 23058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "44d0a2b9561a4e0ab44df49522f7941b"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "682ce1ebf65a87728fc1eea5f3be31d1"
server_key_iv: "59408a64f5d6f76e94ca7266bc0361ef"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.364276 23058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "44d0a2b9561a4e0ab44df49522f7941b"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "682ce1ebf65a87728fc1eea5f3be31d1"
server_key_iv: "59408a64f5d6f76e94ca7266bc0361ef"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.371915 23058 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260929 18:42:13.398756 23075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.408252 23058 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.000s
I20260929 18:42:13.408421 23058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "44d0a2b9561a4e0ab44df49522f7941b"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "682ce1ebf65a87728fc1eea5f3be31d1"
server_key_iv: "59408a64f5d6f76e94ca7266bc0361ef"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.408566 23058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:42:13.435153 23058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:42:13.439227 23058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:42:13.439492 23058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:42:13.440279 23058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:42:13.441402 23058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:42:13.441511 23058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.441659 23058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:42:13.441752 23058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.456046 23058 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:37583
I20260929 18:42:13.457365 23058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:42:13.459075 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 23058
I20260929 18:42:13.459209 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:42:13.459578 11739 external_mini_cluster.cc:1428] Setting key 4206cbc1dc70ad58a5ebc48fd9941bfb
I20260929 18:42:13.458540 23190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:37583 every 8 connection(s)
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:42:13.490054 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:13.463444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:56231 (local address 127.11.118.254:33819)
0929 18:42:13.463717 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:13.463722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:13.468247 (+ 4525us) server_negotiation.cc:407] Connection header received
0929 18:42:13.468304 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:13.468308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:13.468369 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:13.468459 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:42:13.470189 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.471099 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.472884 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.473261 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.473347 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:13.480082 (+ 6735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:42:13.480087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:42:13.480090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:42:13.480124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:42:13.482130 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:13.482910 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:13.482914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:13.482916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:13.483011 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:13.483545 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:13.483549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:13.483551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:13.483811 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:42:13.483899 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:13.489699 (+ 5800us) server_negotiation.cc:299] Negotiation successful
0929 18:42:13.489861 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
I20260929 18:42:13.492343 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:13.462224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33819 (local address 127.11.118.193:56231)
0929 18:42:13.466665 (+ 4441us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:13.466738 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:13.467893 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:13.468809 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:13.468819 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:13.469206 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:42:13.469988 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:13.470009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.471262 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.471265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:13.472737 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:13.472749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.473579 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.473583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:13.473794 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:13.477904 (+ 4110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:42:13.477928 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:42:13.479795 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:42:13.482349 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:13.482356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:13.482378 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:13.482775 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:13.483145 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:13.483148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:13.483150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:13.483412 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:13.483969 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:42:13.483979 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:42:13.484355 (+ 376us) client_negotiation.cc:769] Sending connection context
0929 18:42:13.490616 (+ 6261us) client_negotiation.cc:240] Negotiation successful
0929 18:42:13.491051 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4169,"thread_start_us":127,"threads_started":1}
I20260929 18:42:13.494459 23191 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33819
I20260929 18:42:13.494728 23191 heartbeater.cc:464] Registering TS with master...
I20260929 18:42:13.495402 23191 heartbeater.cc:511] Master 127.11.118.254:33819 requested a full tablet report, sending...
I20260929 18:42:13.497242 22977 ts_manager.cc:194] Registered new tserver with Master: 44d0a2b9561a4e0ab44df49522f7941b (127.11.118.193:37583)
I20260929 18:42:13.498925 22977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:56231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:42:13.526891 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:34231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 18:42:13.709623 23197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:42:13.710165 23197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:42:13.710330 23197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:42:13.717352 23197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:42:13.717526 23197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:42:13.717677 23197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:42:13.726140 23197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.118.212:34231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:42:13.727891 23197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:42:13.729316 23197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:42:13.746603 23204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:13.747233 23203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:13.748943 23197 server_base.cc:1034] running on GCE node
W20260929 18:42:13.750463 23206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:13.751088 23197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:42:13.751988 23197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:42:13.763885 23197 hybrid_clock.cc:648] HybridClock initialized: now 1790707333763792 us; error 122 us; skew 500 ppm
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:42:13.768241 23197 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:42:13.769553 23197 webserver.cc:460] Webserver started at http://127.11.118.194:33291/ using document root <none> and password file <none>
I20260929 18:42:13.770447 23197 fs_manager.cc:359] Metadata directory not provided
I20260929 18:42:13.770632 23197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:42:13.771023 23197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:13.791411 23197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "5b651676e1824ae4984a00a889ab1483"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "c2f76a28019880e959369d1731527fe5"
server_key_iv: "14520d7aa50841bd2629288d0fd23686"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.794042 23197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "5b651676e1824ae4984a00a889ab1483"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "c2f76a28019880e959369d1731527fe5"
server_key_iv: "14520d7aa50841bd2629288d0fd23686"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.799655 23197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 18:42:13.826061 23213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.827884 23197 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:42:13.828042 23197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "5b651676e1824ae4984a00a889ab1483"
format_stamp: "Formatted at 2026-09-29 18:42:13 on dist-test-slave-cgv7"
server_key: "c2f76a28019880e959369d1731527fe5"
server_key_iv: "14520d7aa50841bd2629288d0fd23686"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:13.828202 23197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:42:13.857419 23197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:42:13.862193 23197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:42:13.862531 23197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:42:13.863349 23197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:42:13.864494 23197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:42:13.864602 23197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.864705 23197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:42:13.864769 23197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:13.879316 23197 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:33505
I20260929 18:42:13.880549 23197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:42:13.881444 23326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:33505 every 8 connection(s)
I20260929 18:42:13.884281 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 23197
I20260929 18:42:13.884400 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:42:13.884703 11739 external_mini_cluster.cc:1428] Setting key e8dd40022bb2aac3731cb73d1b7855cf
Sep 29 18:42:13 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:42:13.902375 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:13.884989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38617 (local address 127.11.118.254:33819)
0929 18:42:13.885126 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:13.885131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:13.885971 (+ 840us) server_negotiation.cc:407] Connection header received
0929 18:42:13.888648 (+ 2677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:13.888651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:13.888706 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:13.888784 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:42:13.890088 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.891017 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.892992 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.893435 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.893533 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:13.896813 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:42:13.896820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:42:13.896824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:42:13.896860 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:42:13.898657 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:13.899229 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:13.899234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:13.899236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:13.899320 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:13.899734 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:13.899741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:13.899746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:13.899993 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:42:13.900054 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:13.902033 (+ 1979us) server_negotiation.cc:299] Negotiation successful
0929 18:42:13.902186 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:42:13.904158 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:13.883834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33819 (local address 127.11.118.194:38617)
0929 18:42:13.885621 (+ 1787us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:13.885682 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:13.888397 (+ 2715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:13.888985 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:13.888997 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:13.889413 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:42:13.889938 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:13.889952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.891206 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.891212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:13.892828 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:13.892843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:13.893800 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:13.893807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:13.894046 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:13.894789 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:42:13.894816 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:42:13.896575 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:42:13.898758 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:13.898764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:13.898782 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:13.899119 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:13.899420 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:13.899423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:13.899425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:13.899617 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:13.901374 (+ 1757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:42:13.901382 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:42:13.901745 (+ 363us) client_negotiation.cc:769] Sending connection context
0929 18:42:13.902834 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0929 18:42:13.903181 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1573,"thread_start_us":189,"threads_started":1}
I20260929 18:42:13.914803 23327 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33819
I20260929 18:42:13.915092 23327 heartbeater.cc:464] Registering TS with master...
I20260929 18:42:13.915719 23327 heartbeater.cc:511] Master 127.11.118.254:33819 requested a full tablet report, sending...
I20260929 18:42:13.917222 22977 ts_manager.cc:194] Registered new tserver with Master: 5b651676e1824ae4984a00a889ab1483 (127.11.118.194:33505)
I20260929 18:42:13.917999 22977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:38617
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:42:13.934334 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:34231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:42:14 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
W20260929 18:42:14.093591 23332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:42:14.094080 23332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:42:14.094192 23332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:42:14.100309 23332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:42:14.100483 23332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:42:14.100641 23332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:42:14.108704 23332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.118.212:34231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:42:14.110333 23332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:42:14.111780 23332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:42:14.126921 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:14.126919 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:42:14.127651 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:42:14.128095 23332 server_base.cc:1034] running on GCE node
I20260929 18:42:14.128764 23332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:42:14.129568 23332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:42:14.145171 23332 hybrid_clock.cc:648] HybridClock initialized: now 1790707334145062 us; error 128 us; skew 500 ppm
Sep 29 18:42:14 dist-test-slave-cgv7 krb5kdc[22501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:42:14.149194 23332 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:42:14.150880 23332 webserver.cc:460] Webserver started at http://127.11.118.195:38155/ using document root <none> and password file <none>
I20260929 18:42:14.151643 23332 fs_manager.cc:359] Metadata directory not provided
I20260929 18:42:14.151746 23332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:42:14.152066 23332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:42:14 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:14.175597 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "b2e76a9194514617a3741a372d8ce029"
format_stamp: "Formatted at 2026-09-29 18:42:14 on dist-test-slave-cgv7"
server_key: "0fac3fe3e803695ce9487c0d4d3c16bc"
server_key_iv: "7690b85f7c1aaf4605336838f0b7a65b"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:14.176462 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "b2e76a9194514617a3741a372d8ce029"
format_stamp: "Formatted at 2026-09-29 18:42:14 on dist-test-slave-cgv7"
server_key: "0fac3fe3e803695ce9487c0d4d3c16bc"
server_key_iv: "7690b85f7c1aaf4605336838f0b7a65b"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:14.181522 23332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 18:42:14.200104 23347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:14.201817 23332 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:42:14.201952 23332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "b2e76a9194514617a3741a372d8ce029"
format_stamp: "Formatted at 2026-09-29 18:42:14 on dist-test-slave-cgv7"
server_key: "0fac3fe3e803695ce9487c0d4d3c16bc"
server_key_iv: "7690b85f7c1aaf4605336838f0b7a65b"
server_key_version: "kuduclusterkey@0"
I20260929 18:42:14.202091 23332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:42:14.228621 23332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:42:14.232453 23332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:42:14.232720 23332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:42:14.233491 23332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:42:14.234629 23332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:42:14.234735 23332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:14.234831 23332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:42:14.234894 23332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:42:14.246048 23332 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:33119
I20260929 18:42:14.246198 23460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:33119 every 8 connection(s)
I20260929 18:42:14.247090 23332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:42:14.249495 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 23332
I20260929 18:42:14.249625 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:42:14.249943 11739 external_mini_cluster.cc:1428] Setting key 258615c9c2294376c362562767163c96
Sep 29 18:42:14 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:42:14.261560 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.248940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:53533 (local address 127.11.118.254:33819)
0929 18:42:14.249063 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.249067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.250728 (+ 1661us) server_negotiation.cc:407] Connection header received
0929 18:42:14.252246 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.252249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.252304 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.252383 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:42:14.253451 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.253983 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.255256 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.255487 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.255599 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.257960 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:42:14.257964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:42:14.257966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:42:14.257999 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:42:14.259514 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:14.260059 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:14.260064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:14.260066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:14.260152 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:14.260492 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:14.260497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:14.260499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:14.260679 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:42:14.260793 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.261264 (+ 471us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.261390 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:42:14.262588 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.249441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:33819 (local address 127.11.118.195:53533)
0929 18:42:14.250566 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.250634 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.251920 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.252558 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.252565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.252837 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:42:14.253311 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.253323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.254111 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.254114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.255132 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.255142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.255642 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.255646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.255827 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.256451 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:42:14.256467 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:42:14.257792 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:42:14.259682 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:14.259688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:14.259709 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:14.259952 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:14.260254 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:42:14.260257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:42:14.260259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:42:14.260397 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:42:14.260782 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:42:14.260789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:42:14.261033 (+ 244us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.261224 (+ 191us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.261524 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":141,"threads_started":1}
I20260929 18:42:14.264175 23461 heartbeater.cc:347] Connected to a master server at 127.11.118.254:33819
I20260929 18:42:14.264412 23461 heartbeater.cc:464] Registering TS with master...
I20260929 18:42:14.264848 23461 heartbeater.cc:511] Master 127.11.118.254:33819 requested a full tablet report, sending...
I20260929 18:42:14.265836 22977 ts_manager.cc:194] Registered new tserver with Master: b2e76a9194514617a3741a372d8ce029 (127.11.118.195:33119)
I20260929 18:42:14.266460 22977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:53533
I20260929 18:42:14.276021 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:42:14.287187 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.278351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.11.118.254:33819)
0929 18:42:14.278489 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.278493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.278756 (+ 263us) server_negotiation.cc:407] Connection header received
0929 18:42:14.278929 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.278933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.278991 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.279084 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:42:14.279864 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.280450 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.281386 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.281648 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.281769 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.283376 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:42:14.283381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:42:14.283384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:42:14.283419 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:42:14.285087 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:14.285597 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:14.285603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:14.285605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:14.285696 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:42:14.286003 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:42:14.286009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:42:14.286011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:42:14.286224 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:42:14.286428 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.286856 (+ 428us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.286990 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:42:14.290900 22977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:42:14.294081 22977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:42:14.311478 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.302834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:33505 (local address 127.0.0.1:35558)
0929 18:42:14.304925 (+ 2091us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.304979 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.305127 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.306476 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.306483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.306506 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.306652 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.306660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.307855 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.307858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.309206 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.309215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.310886 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.310889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.310999 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.311036 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.311158 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.311291 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1906,"thread_start_us":108,"threads_started":1}
I20260929 18:42:14.312848 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.303650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.11.118.194:33505)
0929 18:42:14.304117 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.304132 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.305828 (+ 1696us) server_negotiation.cc:407] Connection header received
0929 18:42:14.305920 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.305926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.306171 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.306491 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.306790 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.307649 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.309342 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.310530 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.310662 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.310827 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.312571 (+ 1744us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.312695 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":144,"threads_started":1}
I20260929 18:42:14.316445 23260 tablet_service.cc:1467] Processing CreateTablet for tablet 6e236e6592244cb0ad0856861521cae0 (DEFAULT_TABLE table=test-table [id=3704079a098247b1b367c5b1365b9375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:42:14.317888 23260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e236e6592244cb0ad0856861521cae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:42:14.321691 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.304810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:33119 (local address 127.0.0.1:46626)
0929 18:42:14.310518 (+ 5708us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.310535 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.310640 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.311412 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.311415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.311431 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.312046 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.312055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.317411 (+ 5356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.317418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.318811 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.318825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.320267 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.320271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.320366 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.321356 (+ 990us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.321435 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.321530 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5629,"thread_start_us":77,"threads_started":1}
I20260929 18:42:14.322201 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.305365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.11.118.195:33119)
0929 18:42:14.305668 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.305679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.310745 (+ 5066us) server_negotiation.cc:407] Connection header received
0929 18:42:14.310838 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.310847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.311107 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.311268 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.312180 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.317257 (+ 5077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.319004 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.319920 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.320052 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.320192 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.321913 (+ 1721us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.322037 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":139,"threads_started":1}
I20260929 18:42:14.325554 23395 tablet_service.cc:1467] Processing CreateTablet for tablet 6e236e6592244cb0ad0856861521cae0 (DEFAULT_TABLE table=test-table [id=3704079a098247b1b367c5b1365b9375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:42:14.328876 23395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e236e6592244cb0ad0856861521cae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:42:14.334972 23479 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Bootstrap starting.
I20260929 18:42:14.336220 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.304123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37583 (local address 127.0.0.1:43222)
0929 18:42:14.306750 (+ 2627us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.306764 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.306839 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.331108 (+ 24269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.331112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.331139 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.331263 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.331270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.332910 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.332914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.334284 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.334297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.335784 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.335788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.335887 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.335919 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.335974 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.336058 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2549,"thread_start_us":93,"threads_started":1}
I20260929 18:42:14.337221 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.304296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.11.118.193:37583)
0929 18:42:14.330509 (+ 26213us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.330516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.330547 (+ 31us) server_negotiation.cc:407] Connection header received
0929 18:42:14.330651 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.330659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.330911 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.331437 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.331618 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.332750 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.334450 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.335627 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.336790 (+ 1163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.336931 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.336986 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.337081 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26082,"thread_start_us":87,"threads_started":1}
I20260929 18:42:14.340085 23479 tablet_bootstrap.cc:654] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Neither blocks nor log segments found. Creating new log.
I20260929 18:42:14.340619 23125 tablet_service.cc:1467] Processing CreateTablet for tablet 6e236e6592244cb0ad0856861521cae0 (DEFAULT_TABLE table=test-table [id=3704079a098247b1b367c5b1365b9375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:42:14.341327 23479 log.cc:824] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Log is configured to *not* fsync() on all Append() calls
I20260929 18:42:14.341789 23125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e236e6592244cb0ad0856861521cae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:42:14.346730 23480 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Bootstrap starting.
I20260929 18:42:14.348557 23479 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: No bootstrap required, opened a new log
I20260929 18:42:14.348848 23479 ts_tablet_manager.cc:1397] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260929 18:42:14.349742 23480 tablet_bootstrap.cc:654] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Neither blocks nor log segments found. Creating new log.
I20260929 18:42:14.353070 23479 raft_consensus.cc:348] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.353593 23479 raft_consensus.cc:374] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:42:14.353716 23479 raft_consensus.cc:729] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b651676e1824ae4984a00a889ab1483, State: Initialized, Role: FOLLOWER
I20260929 18:42:14.354262 23479 consensus_queue.cc:260] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.355412 23479 ts_tablet_manager.cc:1428] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:42:14.355813 23327 heartbeater.cc:503] Master 127.11.118.254:33819 was elected leader, sending a full tablet report...
I20260929 18:42:14.359481 23482 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Bootstrap starting.
I20260929 18:42:14.359483 23480 log.cc:824] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Log is configured to *not* fsync() on all Append() calls
I20260929 18:42:14.361609 23482 tablet_bootstrap.cc:654] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Neither blocks nor log segments found. Creating new log.
I20260929 18:42:14.361847 23480 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: No bootstrap required, opened a new log
I20260929 18:42:14.362073 23480 ts_tablet_manager.cc:1397] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260929 18:42:14.362483 23482 log.cc:824] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:42:14.366227 23480 raft_consensus.cc:348] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.366664 23480 raft_consensus.cc:374] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:42:14.366771 23480 raft_consensus.cc:729] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e76a9194514617a3741a372d8ce029, State: Initialized, Role: FOLLOWER
I20260929 18:42:14.367009 23482 tablet_bootstrap.cc:492] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: No bootstrap required, opened a new log
I20260929 18:42:14.367211 23482 ts_tablet_manager.cc:1397] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260929 18:42:14.367463 23480 consensus_queue.cc:260] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.368697 23480 ts_tablet_manager.cc:1428] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260929 18:42:14.369149 23461 heartbeater.cc:503] Master 127.11.118.254:33819 was elected leader, sending a full tablet report...
I20260929 18:42:14.372655 23482 raft_consensus.cc:348] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.372936 23482 raft_consensus.cc:374] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:42:14.372983 23482 raft_consensus.cc:729] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d0a2b9561a4e0ab44df49522f7941b, State: Initialized, Role: FOLLOWER
I20260929 18:42:14.373435 23482 consensus_queue.cc:260] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.374182 23191 heartbeater.cc:503] Master 127.11.118.254:33819 was elected leader, sending a full tablet report...
I20260929 18:42:14.374643 23482 ts_tablet_manager.cc:1428] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260929 18:42:14.383713 23328 tablet.cc:2369] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:42:14.389695 23486 raft_consensus.cc:482] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:42:14.389916 23486 raft_consensus.cc:504] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.395471 23486 leader_election.cc:288] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b651676e1824ae4984a00a889ab1483 (127.11.118.194:33505), 44d0a2b9561a4e0ab44df49522f7941b (127.11.118.193:37583)
I20260929 18:42:14.396914 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.391967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:33505 (local address 127.11.118.195:46729)
0929 18:42:14.392281 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.392299 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.392408 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.393075 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.393078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.393099 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.393227 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.393234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.394304 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.394308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.395528 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.395539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.396466 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.396470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.396577 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.396605 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.396697 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.396775 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260929 18:42:14.396966 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.392603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:46729 (local address 127.11.118.194:33505)
0929 18:42:14.392737 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.392742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.392757 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:42:14.392811 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.392813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.392872 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.392946 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.393367 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.394110 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.395722 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.396334 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.396456 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.396513 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.396746 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.396843 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:42:14.397785 23265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "b2e76a9194514617a3741a372d8ce029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b651676e1824ae4984a00a889ab1483" is_pre_election: true
I20260929 18:42:14.398121 23265 raft_consensus.cc:2454] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e76a9194514617a3741a372d8ce029 in term 0.
I20260929 18:42:14.399247 23351 leader_election.cc:302] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b651676e1824ae4984a00a889ab1483, b2e76a9194514617a3741a372d8ce029; no voters:
I20260929 18:42:14.399569 23486 raft_consensus.cc:2781] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:42:14.399652 23486 raft_consensus.cc:482] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:42:14.399704 23486 raft_consensus.cc:3037] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:42:14.401405 23486 raft_consensus.cc:504] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.401810 23486 leader_election.cc:288] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [CANDIDATE]: Term 1 election: Requested vote from peers 5b651676e1824ae4984a00a889ab1483 (127.11.118.194:33505), 44d0a2b9561a4e0ab44df49522f7941b (127.11.118.193:37583)
I20260929 18:42:14.401891 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.395284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:59771 (local address 127.11.118.193:37583)
0929 18:42:14.395392 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.395396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.396160 (+ 764us) server_negotiation.cc:407] Connection header received
0929 18:42:14.397545 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.397549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.397610 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.397664 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.398075 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.398869 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.400470 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.401097 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.401221 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.401277 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.401665 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.401756 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:42:14.402411 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.395201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37583 (local address 127.11.118.195:59771)
0929 18:42:14.396072 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.396087 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.396232 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.397800 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.397802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.397819 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.397938 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.397944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.399149 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.399152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.400313 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.400328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.401409 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.401413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.401512 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.401536 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.402094 (+ 558us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.402218 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":123,"threads_started":1}
I20260929 18:42:14.402638 23265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "b2e76a9194514617a3741a372d8ce029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b651676e1824ae4984a00a889ab1483"
I20260929 18:42:14.402772 23265 raft_consensus.cc:3037] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:42:14.403312 23487 raft_consensus.cc:482] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:42:14.403345 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "b2e76a9194514617a3741a372d8ce029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d0a2b9561a4e0ab44df49522f7941b" is_pre_election: true
I20260929 18:42:14.403475 23487 raft_consensus.cc:504] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.403705 23141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "b2e76a9194514617a3741a372d8ce029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d0a2b9561a4e0ab44df49522f7941b"
I20260929 18:42:14.404002 23265 raft_consensus.cc:2454] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e76a9194514617a3741a372d8ce029 in term 1.
I20260929 18:42:14.404178 23140 raft_consensus.cc:2454] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e76a9194514617a3741a372d8ce029 in term 0.
I20260929 18:42:14.404394 23351 leader_election.cc:302] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b651676e1824ae4984a00a889ab1483, b2e76a9194514617a3741a372d8ce029; no voters:
I20260929 18:42:14.404574 23486 raft_consensus.cc:2781] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:42:14.404639 23487 leader_election.cc:288] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b651676e1824ae4984a00a889ab1483 (127.11.118.194:33505), b2e76a9194514617a3741a372d8ce029 (127.11.118.195:33119)
I20260929 18:42:14.404799 23486 raft_consensus.cc:686] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 1 LEADER]: Becoming Leader. State: Replica: b2e76a9194514617a3741a372d8ce029, State: Running, Role: LEADER
I20260929 18:42:14.405146 23486 consensus_queue.cc:237] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } }
I20260929 18:42:14.409381 22976 catalog_manager.cc:5495] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e76a9194514617a3741a372d8ce029 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "b2e76a9194514617a3741a372d8ce029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e76a9194514617a3741a372d8ce029" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33119 } health_report { overall_health: HEALTHY } } }
I20260929 18:42:14.417675 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.406041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:33505 (local address 127.11.118.193:38015)
0929 18:42:14.410598 (+ 4557us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.410623 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.410735 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.413277 (+ 2542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.413280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.413302 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.413450 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.413458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.414612 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.414654 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.416173 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.416188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.417200 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.417204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.417305 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.417328 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.417400 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.417471 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4482,"thread_start_us":114,"threads_started":1}
I20260929 18:42:14.418088 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.408246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:38015 (local address 127.11.118.194:33505)
0929 18:42:14.408371 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.408376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.412344 (+ 3968us) server_negotiation.cc:407] Connection header received
0929 18:42:14.412402 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.412405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.412454 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.412547 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.413617 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.414460 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.416335 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.416962 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.417082 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.417137 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.417865 (+ 728us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.417946 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:42:14.418756 23265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "44d0a2b9561a4e0ab44df49522f7941b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b651676e1824ae4984a00a889ab1483" is_pre_election: true
I20260929 18:42:14.418972 23265 raft_consensus.cc:2379] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 44d0a2b9561a4e0ab44df49522f7941b in current term 1: Already voted for candidate b2e76a9194514617a3741a372d8ce029 in this term.
I20260929 18:42:14.419109 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.406410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:33119 (local address 127.11.118.193:37385)
0929 18:42:14.410941 (+ 4531us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.410952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.411021 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.414793 (+ 3772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:42:14.414796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:42:14.414820 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:42:14.414970 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.414977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.415960 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.415963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.417119 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:42:14.417130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.418678 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.418681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:42:14.418781 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.418807 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:42:14.418883 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 18:42:14.418954 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4478,"mutex_wait_us":22,"thread_start_us":81,"threads_started":1}
I20260929 18:42:14.419884 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.406803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:37385 (local address 127.11.118.195:33119)
0929 18:42:14.406917 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.406921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.412041 (+ 5120us) server_negotiation.cc:407] Connection header received
0929 18:42:14.412102 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.412105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.412154 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.412242 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:42:14.415123 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.415823 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.417274 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.417832 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.417927 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.417973 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.419660 (+ 1687us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.419764 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 18:42:14.420478 23415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e236e6592244cb0ad0856861521cae0" candidate_uuid: "44d0a2b9561a4e0ab44df49522f7941b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e76a9194514617a3741a372d8ce029" is_pre_election: true
I20260929 18:42:14.421211 23079 leader_election.cc:302] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44d0a2b9561a4e0ab44df49522f7941b; no voters: 5b651676e1824ae4984a00a889ab1483, b2e76a9194514617a3741a372d8ce029
I20260929 18:42:14.421779 23487 raft_consensus.cc:3037] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:42:14.423108 23487 raft_consensus.cc:2726] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Sep 29 18:42:14 dist-test-slave-cgv7 krb5kdc[22501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:14.442394 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.435877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.11.118.195:33119)
0929 18:42:14.436002 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:42:14.436006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:42:14.436018 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:42:14.436103 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:42:14.436109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:42:14.436185 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:42:14.436285 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:42:14.438380 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.439161 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.440493 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:42:14.440833 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:42:14.440931 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:42:14.441300 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:42:14.441871 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:42:14.441983 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 18:42:14.442137 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 18:42:14.442212 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:42:14.450475 23140 raft_consensus.cc:1261] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b [term 1 FOLLOWER]: Refusing update from remote peer b2e76a9194514617a3741a372d8ce029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:42:14.451361 23490 consensus_queue.cc:1035] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d0a2b9561a4e0ab44df49522f7941b" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:42:14.451606 23265 raft_consensus.cc:1261] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 1 FOLLOWER]: Refusing update from remote peer b2e76a9194514617a3741a372d8ce029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:42:14.452143 23490 consensus_queue.cc:1035] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b651676e1824ae4984a00a889ab1483" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 33505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:42:14.461187 23192 tablet.cc:2369] T 6e236e6592244cb0ad0856861521cae0 P 44d0a2b9561a4e0ab44df49522f7941b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:42:14.480926 23495 mvcc.cc:205] Tried to move back new op lower bound from 7334737241899765760 to 7334737241726423040. Current Snapshot: MvccSnapshot[applied={T|T < 7334737241899765760}]
I20260929 18:42:14.481635 23496 mvcc.cc:205] Tried to move back new op lower bound from 7334737241899765760 to 7334737241726423040. Current Snapshot: MvccSnapshot[applied={T|T < 7334737241899765760}]
I20260929 18:42:14.488318 23497 mvcc.cc:205] Tried to move back new op lower bound from 7334737241899765760 to 7334737241726423040. Current Snapshot: MvccSnapshot[applied={T|T < 7334737241899765760}]
I20260929 18:42:14.497440 22976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:42:14.497641 22976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:42:14.502492 22976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6721aba5de0402f92ea9f221f1ca68d: Sending DeleteTablet for 3 replicas of tablet 6e236e6592244cb0ad0856861521cae0
I20260929 18:42:14.503553 23260 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e236e6592244cb0ad0856861521cae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:42:14 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:35558
I20260929 18:42:14.504704 23125 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e236e6592244cb0ad0856861521cae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:42:14 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:43222
I20260929 18:42:14.505782 23395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e236e6592244cb0ad0856861521cae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:42:14 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46626
I20260929 18:42:14.507320 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 23058
I20260929 18:42:14.508100 23504 tablet_replica.cc:330] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483: stopping tablet replica
I20260929 18:42:14.508450 23504 raft_consensus.cc:2229] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:42:14.508764 23504 raft_consensus.cc:2258] T 6e236e6592244cb0ad0856861521cae0 P 5b651676e1824ae4984a00a889ab1483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:42:14.510480 23506 tablet_replica.cc:330] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: stopping tablet replica
I20260929 18:42:14.510772 23506 raft_consensus.cc:2229] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:42:14.511090 23506 raft_consensus.cc:2258] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 18:42:14.518796 22962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:37583 (error 108)
I20260929 18:42:14.519886 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 23197
I20260929 18:42:14.520803 23506 ts_tablet_manager.cc:1905] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:42:14.524940 23506 ts_tablet_manager.cc:1918] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:42:14.525133 23506 log.cc:1196] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6e236e6592244cb0ad0856861521cae0
I20260929 18:42:14.525563 23506 ts_tablet_manager.cc:1939] T 6e236e6592244cb0ad0856861521cae0 P b2e76a9194514617a3741a372d8ce029: Deleting consensus metadata
I20260929 18:42:14.526984 22964 catalog_manager.cc:4841] TS b2e76a9194514617a3741a372d8ce029 (127.11.118.195:33119): tablet 6e236e6592244cb0ad0856861521cae0 (table test-table [id=3704079a098247b1b367c5b1365b9375]) successfully deleted
I20260929 18:42:14.528009 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:42:14.527691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37583 (local address 127.0.0.1:43234)
0929 18:42:14.527856 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.527921 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260929 18:42:14.528282 22962 catalog_manager.cc:4584] TS 44d0a2b9561a4e0ab44df49522f7941b (127.11.118.193:37583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e236e6592244cb0ad0856861521cae0: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37583: connect: Connection refused (error 111)
W20260929 18:42:14.532071 23477 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:42:14.531553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:33505 (local address 127.0.0.1:35562)
0929 18:42:14.531699 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 18:42:14.531713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:42:14.531792 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:42:14.531977 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.194:33505: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 18:42:14.532732 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 23332
I20260929 18:42:14.543712 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 22945
I20260929 18:42:14.555932 11739 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 18:42:14.900964 11739 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 18:42:14.901087 11739 mini_ranger.cc:66] Stopping Ranger...
I20260929 18:42:15.246531 11739 mini_ranger.cc:68] Stopped Ranger
2026-09-29 18:42:15.246 UTC [22531] LOG: received smart shutdown request
2026-09-29 18:42:15.250 UTC [22531] LOG: background worker "logical replication launcher" (PID 22540) exited with exit code 1
2026-09-29 18:42:15.250 UTC [22535] LOG: shutting down
2026-09-29 18:42:15.431 UTC [22531] LOG: database system is shut down
2026-09-29T18:42:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48282 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:42:15 dist-test-slave-cgv7 krb5kdc[23517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:42:15 dist-test-slave-cgv7 krb5kdc[23517](info): set up 2 sockets
Sep 29 18:42:15 dist-test-slave-cgv7 krb5kdc[23517](info): commencing operation
krb5kdc: starting...
W20260929 18:42:16.598387 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:42:16 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:42:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangeradmin_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangeradmin_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangerlookup_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangerlookup_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
I20260929 18:42:16.773427 11739 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqumnYE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/postgres -l logfile start
2026-09-29 18:42:19.465 UTC [23547] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 18:42:19.465 UTC [23547] LOG: listening on IPv4 address "127.11.118.212", port 41841
2026-09-29 18:42:19.470 UTC [23547] LOG: listening on Unix socket "/tmp/.s.PGSQL.41841"
2026-09-29 18:42:19.506 UTC [23550] LOG: database system was shut down at 2026-09-29 18:42:17 UTC
2026-09-29 18:42:19.511 UTC [23547] LOG: database system is ready to accept connections
I20260929 18:42:20.470180 11739 mini_postgres.cc:96] Postgres bound to 41841
2026-09-29 18:42:20.476 UTC [23558] FATAL: database "slave" does not exist
127.11.118.212:41841 - accepting connections
I20260929 18:42:20.476994 11739 mini_ranger.cc:161] Starting Ranger...
I20260929 18:42:20.492393 11739 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 18:42:21.015347 11739 mini_ranger.cc:87] Created ranger Postgres database
I20260929 18:42:21.015444 11739 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin
2026-09-29 18:42:21,277 [I] DB FLAVOR :POSTGRES
2026-09-29 18:42:21,278 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 18:42:21,278 [I] Checking connection..
2026-09-29 18:42:21,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 18:42:21,534 [I] Checking connection passed.
2026-09-29 18:42:21,534 [I] --------- Verifying version history table ---------
2026-09-29 18:42:21,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:42:21,801 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 18:42:21,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:42:22,037 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 18:42:22,037 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 18:42:22,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 18:42:22.284 UTC [23622] WARNING: there is no transaction in progress
2026-09-29 18:42:22,292 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 18:42:22,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 18:42:22,559 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 18:42:22,560 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 18:42:22,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 18:42:22,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 18:42:23,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cgv7.c.gcp-upstream.internal','N') ;"
2026-09-29 18:42:23,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 18:42:23,537 [I] Table x_portal_user does not exist in database ranger
2026-09-29 18:42:23,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 18:42:23,780 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 18:42:23,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 18:42:24,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 18:42:24,295 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 18:42:24,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 18:42:25.354 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.374 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.393 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.410 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.427 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.448 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.519 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.526 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.542 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.553 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.564 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.580 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.590 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.598 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.606 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:25.613 UTC [23730] WARNING: there is no transaction in progress
2026-09-29 18:42:26,325 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 18:42:26,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 18:42:26,567 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 18:42:26,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 18:42:26,816 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 18:42:26,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 18:42:27,066 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 18:42:27,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 18:42:27,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 18:42:27,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cgv7.c.gcp-upstream.internal';"
2026-09-29 18:42:27,802 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 18:42:27,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 18:42:28,048 [I] DB_PATCHES have already been applied
I20260929 18:42:28.054649 11739 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqumnYE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 18:42:28.054740 11739 mini_ranger.cc:193] Using host: 127.11.118.212
I20260929 18:42:28.058338 11739 mini_ranger.cc:227] Ranger admin URL: http://127.11.118.212:56291
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 6:42:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 18:42:29.160502 11739 mini_ranger.cc:160] Time spent starting Ranger: real 8.684s user 0.009s sys 0.001s
Sep 29, 2026 6:42:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 6:42:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 29, 2026 6:42:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 6:42:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 6:42:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 18:42:51.410477 11739 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.118.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.118.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangerkms_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.118.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/rangerkms_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.118.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.118.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.118.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/HTTP_127.11.118.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 18:42:51 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707371, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 18:42:51.503185 11739 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 18:42:51.513600 11739 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 18:42:52.463667 11739 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 18:42:52.463757 11739 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms
I20260929 18:42:52.463776 11739 mini_ranger_kms.cc:227] Using postgres at 127.11.118.212:41841
2026-09-29 18:42:52,645 [I] DB FLAVOR :POSTGRES
2026-09-29 18:42:52,645 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 18:42:52,645 [I] Checking connection
2026-09-29 18:42:52,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 18:42:52,883 [I] connection success
2026-09-29 18:42:52,884 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 18:42:52,884 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 18:42:52,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 18:42:53,152 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 18:42:53,153 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 18:42:53,153 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 18:42:53,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.118.212:41841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 18:42:53,418 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 18:42:53.945394 11739 mini_ranger_kms.cc:310] Created kms service
I20260929 18:42:54.134187 11739 mini_ranger_kms.cc:326] Created kudu user
I20260929 18:42:54.178692 11739 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 18:42:54 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707371, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:42:54.598613 11739 mini_ranger_kms.cc:384] Added ranger policy
I20260929 18:42:54.598788 11739 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqumnYE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqumnYE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 18:42:54.598821 11739 mini_ranger_kms.cc:240] Using host: 127.11.118.212
I20260929 18:42:54.602696 11739 mini_ranger_kms.cc:276] Ranger KMS PID: 23910
I20260929 18:42:54.602795 11739 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.118.212:39337
Sep 29, 2026 6:42:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:42:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:42:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 6:42:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 6:42:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 6:42:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 29 18:42:56 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790707376, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:42:56.914951 11739 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.412s user 0.028s sys 0.136s
I20260929 18:42:56.915078 11739 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 18:42:56.915154 11739 mini_ranger_kms.cc:401] 127.11.118.212:39337/kms/v1/keys
Sep 29 18:42:57 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707376, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
Sep 29 18:42:57 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707376, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.118.212@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
Sep 29 18:42:59 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:42:59.990568 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46441
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:46441
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:00.107187 23961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:00.107585 23961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:00.107734 23961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:00.112130 23961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:00.112206 23961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:00.112226 23961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:00.112243 23961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:00.112284 23961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:00.117965 23961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.118.212:39337/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:46441
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:00.119067 23961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:00.120081 23961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:00.129700 23966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:00.129603 23967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:00.129746 23961 server_base.cc:1034] running on GCE node
W20260929 18:43:00.129602 23969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:00.130630 23961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:00.131769 23961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:00.132967 23961 hybrid_clock.cc:648] HybridClock initialized: now 1790707380132940 us; error 55 us; skew 500 ppm
Sep 29 18:43:00 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:00.140115 23961 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:00.141165 23961 webserver.cc:460] Webserver started at http://127.11.118.254:33201/ using document root <none> and password file <none>
I20260929 18:43:00.141741 23961 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:00.141796 23961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:00.142024 23961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:43:00 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:00.198143 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "f70ba5e42cd1438695835e9f3ddc8c3b"
format_stamp: "Formatted at 2026-09-29 18:43:00 on dist-test-slave-cgv7"
server_key: "6b8bdd3af60cf8d7ba2cc19fb1a76cdf"
server_key_iv: "ec38e16b64a46ba550d318b438cb8945"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:00.198897 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "f70ba5e42cd1438695835e9f3ddc8c3b"
format_stamp: "Formatted at 2026-09-29 18:43:00 on dist-test-slave-cgv7"
server_key: "6b8bdd3af60cf8d7ba2cc19fb1a76cdf"
server_key_iv: "ec38e16b64a46ba550d318b438cb8945"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:00.203270 23961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:00.221408 23976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:00.223026 23961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 18:43:00.223171 23961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f70ba5e42cd1438695835e9f3ddc8c3b"
format_stamp: "Formatted at 2026-09-29 18:43:00 on dist-test-slave-cgv7"
server_key: "6b8bdd3af60cf8d7ba2cc19fb1a76cdf"
server_key_iv: "ec38e16b64a46ba550d318b438cb8945"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:00.223305 23961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:00.242616 23961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:00.246318 23961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:00.246543 23961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:00.255868 23961 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:46441
I20260929 18:43:00.255865 24028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:46441 every 8 connection(s)
I20260929 18:43:00.257145 23961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:00.258883 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 23961
I20260929 18:43:00.259027 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:43:00.259337 11739 external_mini_cluster.cc:1428] Setting key 41a1f710dc26d2fd9006ebb59b8d46f5
I20260929 18:43:00.261425 24029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:00.267679 24029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Bootstrap starting.
Sep 29 18:43:00 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:00.270692 24029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:00.271633 24029 log.cc:824] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:00.274178 24029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: No bootstrap required, opened a new log
I20260929 18:43:00.277307 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:00.260795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.11.118.254:46441)
0929 18:43:00.261224 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:00.261243 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:00.261288 (+ 45us) server_negotiation.cc:407] Connection header received
0929 18:43:00.262198 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:00.262224 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:00.262693 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:00.263103 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:00.263948 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:00.265521 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:00.266819 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:00.267656 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:00.267830 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:00.271199 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:00.271206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:00.271223 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:00.271257 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:00.273973 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:00.274549 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:00.274557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:00.274565 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:00.274701 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:00.275037 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:00.275040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:00.275042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:00.275538 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:00.275721 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:00.276060 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 18:43:00.276341 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260929 18:43:00.279052 24029 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:00.279323 24029 raft_consensus.cc:374] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:00.279421 24029 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70ba5e42cd1438695835e9f3ddc8c3b, State: Initialized, Role: FOLLOWER
I20260929 18:43:00.279922 24029 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:00.280078 24029 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:00.280141 24029 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:00.280233 24029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:00.281186 24029 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:00.281533 24029 leader_election.cc:302] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70ba5e42cd1438695835e9f3ddc8c3b; no voters:
I20260929 18:43:00.281889 24029 leader_election.cc:288] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:43:00.281971 24034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:00.282363 24034 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 LEADER]: Becoming Leader. State: Replica: f70ba5e42cd1438695835e9f3ddc8c3b, State: Running, Role: LEADER
I20260929 18:43:00.282681 24034 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:00.283252 24029 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:00.283847 24034 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } } }
I20260929 18:43:00.284013 24034 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:00.284264 24034 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70ba5e42cd1438695835e9f3ddc8c3b. Latest consensus state: current_term: 1 leader_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } } }
I20260929 18:43:00.284381 24034 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:00.285604 24029 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 18:43:00.286844 24041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:00.293367 24041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:00.300908 24041 catalog_manager.cc:1347] Generated new cluster ID: 40df884e6f364fd3a89cf9ac91c9bbe9
I20260929 18:43:00.300987 24041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:00.337354 24041 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:43:00.338809 24041 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:00.344528 24041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Generated new TSK 0
I20260929 18:43:00.345477 24041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:43:01.075570 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:01.237828 24074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:01.238114 24074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:01.238207 24074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:01.244894 24074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:01.244985 24074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:01.245093 24074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:01.251470 24074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:01.252872 24074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:01.254330 24074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:01.266623 24082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.266788 24081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.266721 24084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:01.268168 24074 server_base.cc:1034] running on GCE node
I20260929 18:43:01.268779 24074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:01.269577 24074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:01.270772 24074 hybrid_clock.cc:648] HybridClock initialized: now 1790707381270744 us; error 44 us; skew 500 ppm
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:01.274493 24074 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:01.275642 24074 webserver.cc:460] Webserver started at http://127.11.118.193:41207/ using document root <none> and password file <none>
I20260929 18:43:01.276400 24074 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:01.276495 24074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:01.276715 24074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:01.293346 24074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "bcfd84d2f43f4c55b0b5e5633e042a91"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "f0676cf6b33e21024c6d64e27509f60f"
server_key_iv: "a362b74c07fde07a4a533841bc093de0"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.294198 24074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "bcfd84d2f43f4c55b0b5e5633e042a91"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "f0676cf6b33e21024c6d64e27509f60f"
server_key_iv: "a362b74c07fde07a4a533841bc093de0"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.299654 24074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 18:43:01.325709 24091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.327546 24074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:43:01.327687 24074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "bcfd84d2f43f4c55b0b5e5633e042a91"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "f0676cf6b33e21024c6d64e27509f60f"
server_key_iv: "a362b74c07fde07a4a533841bc093de0"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.327822 24074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:01.357518 24074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:01.361455 24074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:01.361732 24074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:01.362591 24074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:01.363722 24074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:01.363827 24074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.363965 24074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:01.364044 24074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.374998 24074 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:36851
I20260929 18:43:01.376194 24074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:01.381273 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24074
I20260929 18:43:01.381403 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:43:01.381691 11739 external_mini_cluster.cc:1428] Setting key da4d46dc99140b2866474ec85f23dc25
I20260929 18:43:01.390570 24206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:36851 every 8 connection(s)
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:43:01.418593 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:01.393395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.193:38831)
0929 18:43:01.395095 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:01.395150 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:01.396183 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:01.401788 (+ 5605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:01.401800 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:01.402208 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:01.402981 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:01.403004 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.404204 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.404208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:01.405688 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:01.405701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.406570 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.406574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:01.406784 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:01.407585 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:01.407608 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:01.412513 (+ 4905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:01.415028 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:01.415036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:01.415056 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:01.415447 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:01.415766 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:01.415769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:01.415771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:01.415983 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:01.416466 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:01.416474 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:01.416820 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 18:43:01.417049 (+ 229us) client_negotiation.cc:240] Negotiation successful
0929 18:43:01.417420 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":177,"threads_started":1}
I20260929 18:43:01.419380 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:01.392870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:38831 (local address 127.11.118.254:46441)
0929 18:43:01.395694 (+ 2824us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:01.395699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:01.395719 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:43:01.401172 (+ 5453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:01.401177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:01.401256 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:01.401373 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:01.403183 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.404040 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.405857 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.406200 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.406350 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:01.412797 (+ 6447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:01.412801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:01.412804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:01.412839 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:01.414866 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:01.415573 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:01.415577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:01.415579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:01.415669 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:01.416090 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:01.416093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:01.416095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:01.416323 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:01.416404 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:01.419076 (+ 2672us) server_negotiation.cc:299] Negotiation successful
0929 18:43:01.419218 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2732,"thread_start_us":103,"threads_started":1}
I20260929 18:43:01.421319 24207 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:01.421572 24207 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:01.422163 24207 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:01.423980 23993 ts_manager.cc:194] Registered new tserver with Master: bcfd84d2f43f4c55b0b5e5633e042a91 (127.11.118.193:36851)
I20260929 18:43:01.425551 23993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:38831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:01.446581 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:01.604815 24213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:01.605125 24213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:01.605240 24213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:01.610178 24213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:01.610342 24213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:01.610491 24213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:01.617403 24213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:01.618610 24213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:01.619660 24213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:01.631189 24222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.631408 24219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.631524 24220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:01.631949 24213 server_base.cc:1034] running on GCE node
I20260929 18:43:01.632522 24213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:01.633265 24213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:01.634457 24213 hybrid_clock.cc:648] HybridClock initialized: now 1790707381634418 us; error 49 us; skew 500 ppm
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:01.638017 24213 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:01.638954 24213 webserver.cc:460] Webserver started at http://127.11.118.194:38873/ using document root <none> and password file <none>
I20260929 18:43:01.639540 24213 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:01.639633 24213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:01.639914 24213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:01.658872 24213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "fba29dc0b67d467c8346eb1e75530a08"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "c285d277eec76abcb5ef723c48bbec14"
server_key_iv: "b902c83245d0e5479aa39cbd035a2146"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.660008 24213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "fba29dc0b67d467c8346eb1e75530a08"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "c285d277eec76abcb5ef723c48bbec14"
server_key_iv: "b902c83245d0e5479aa39cbd035a2146"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.667908 24213 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260929 18:43:01.695230 24229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.696995 24213 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260929 18:43:01.697132 24213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "fba29dc0b67d467c8346eb1e75530a08"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "c285d277eec76abcb5ef723c48bbec14"
server_key_iv: "b902c83245d0e5479aa39cbd035a2146"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:01.697258 24213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:01.712908 24213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:01.721455 24213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:01.721771 24213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:01.722623 24213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:01.723879 24213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:01.723949 24213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.724009 24213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:01.724044 24213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:01.735468 24213 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:43059
I20260929 18:43:01.736723 24213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:01.743278 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24213
I20260929 18:43:01.743417 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:43:01.743741 11739 external_mini_cluster.cc:1428] Setting key e8aff85dc4ed40969fc558166291c63e
I20260929 18:43:01.751713 24342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:43059 every 8 connection(s)
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:01.771452 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:01.754879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:42265 (local address 127.11.118.254:46441)
0929 18:43:01.755059 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:01.755064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:01.755507 (+ 443us) server_negotiation.cc:407] Connection header received
0929 18:43:01.757055 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:01.757064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:01.757139 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:01.757203 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:01.758881 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.759845 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.761684 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.762125 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.762215 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:01.766991 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:01.766998 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:01.767017 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:01.767057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:01.768833 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:01.769440 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:01.769443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:01.769444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:01.769525 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:01.769906 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:01.769910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:01.769911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:01.770151 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:01.770265 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:01.771068 (+ 803us) server_negotiation.cc:299] Negotiation successful
0929 18:43:01.771233 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 18:43:01.772969 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:01.754242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.194:42265)
0929 18:43:01.755344 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:01.755402 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:01.756707 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:01.757671 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:01.757682 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:01.757978 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:01.758679 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:01.758704 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.760026 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.760032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:01.761527 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:01.761541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:01.762452 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:01.762459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:01.762690 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:01.763424 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:01.763441 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:01.766732 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:01.768981 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:01.768989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:01.769009 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:01.769311 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:01.769625 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:01.769628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:01.769630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:01.769799 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:01.770299 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:01.770310 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:01.770727 (+ 417us) client_negotiation.cc:769] Sending connection context
0929 18:43:01.771757 (+ 1030us) client_negotiation.cc:240] Negotiation successful
0929 18:43:01.772074 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":181,"threads_started":1}
I20260929 18:43:01.774256 24343 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:01.774559 24343 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:01.775077 24343 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:01.776129 23993 ts_manager.cc:194] Registered new tserver with Master: fba29dc0b67d467c8346eb1e75530a08 (127.11.118.194:43059)
I20260929 18:43:01.776708 23993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:42265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:43:01.797920 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:01.942240 24348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:01.942689 24348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:01.942775 24348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:01.949487 24348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:01.949572 24348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:01.949669 24348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:01.956164 24348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:01.957433 24348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:01.958956 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:01.971657 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.981413 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:01.982048 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:01.985100 24348 server_base.cc:1034] running on GCE node
I20260929 18:43:01.985741 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:01.986717 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:01.987892 24348 hybrid_clock.cc:648] HybridClock initialized: now 1790707381987863 us; error 47 us; skew 500 ppm
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:01.991330 24348 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:01.992527 24348 webserver.cc:460] Webserver started at http://127.11.118.195:41463/ using document root <none> and password file <none>
I20260929 18:43:01.993270 24348 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:01.993404 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:01.993690 24348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 18:43:01 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:02.029850 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "3d46661b65e44ea5b8f78537524c775d"
format_stamp: "Formatted at 2026-09-29 18:43:02 on dist-test-slave-cgv7"
server_key: "09a9c25a37331e44eb373f93ba332826"
server_key_iv: "8cdd97f6b5e993e84dc7617321409b8a"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.030851 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "3d46661b65e44ea5b8f78537524c775d"
format_stamp: "Formatted at 2026-09-29 18:43:02 on dist-test-slave-cgv7"
server_key: "09a9c25a37331e44eb373f93ba332826"
server_key_iv: "8cdd97f6b5e993e84dc7617321409b8a"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.037029 24348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260929 18:43:02.065771 24363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.067785 24348 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.003s sys 0.000s
I20260929 18:43:02.067956 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "3d46661b65e44ea5b8f78537524c775d"
format_stamp: "Formatted at 2026-09-29 18:43:02 on dist-test-slave-cgv7"
server_key: "09a9c25a37331e44eb373f93ba332826"
server_key_iv: "8cdd97f6b5e993e84dc7617321409b8a"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.068105 24348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:02.085544 24348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:02.089299 24348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:02.089586 24348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:02.090312 24348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:02.091440 24348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:02.091544 24348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.091639 24348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:02.091699 24348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.105082 24348 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:35435
I20260929 18:43:02.106518 24348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:02.107146 24476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:35435 every 8 connection(s)
I20260929 18:43:02.109927 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24348
I20260929 18:43:02.110070 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:43:02.110422 11739 external_mini_cluster.cc:1428] Setting key 2383e8701d19346ec11d15b99019020c
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707381, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:02.122315 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.109498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:55843 (local address 127.11.118.254:46441)
0929 18:43:02.109628 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:02.109632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:02.110999 (+ 1367us) server_negotiation.cc:407] Connection header received
0929 18:43:02.112019 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:02.112022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:02.112079 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:02.112164 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:02.113254 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.113794 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.115969 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.116230 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.116290 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.118741 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:02.118744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:02.118746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:02.118769 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:02.120229 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.120759 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.120762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.120764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.120844 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.121206 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.121209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.121210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.121384 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:02.121489 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:02.121980 (+ 491us) server_negotiation.cc:299] Negotiation successful
0929 18:43:02.122109 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:43:02.123149 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.109951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.195:55843)
0929 18:43:02.110617 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:02.110670 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:02.111794 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:02.112354 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:02.112361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:02.112625 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:02.113106 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.113119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.113938 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.113940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.115823 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.115834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.116450 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.116452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.116585 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.117199 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:02.117215 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:02.118575 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:02.120362 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.120367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.120383 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.120648 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.120951 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.120954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.120956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.121103 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.121488 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:02.121495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:02.121736 (+ 241us) client_negotiation.cc:769] Sending connection context
0929 18:43:02.121919 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 18:43:02.122219 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":185,"threads_started":1}
I20260929 18:43:02.124245 24477 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:02.124471 24477 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.124953 24477 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:02.125872 23993 ts_manager.cc:194] Registered new tserver with Master: 3d46661b65e44ea5b8f78537524c775d (127.11.118.195:35435)
I20260929 18:43:02.126598 23993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:55843
I20260929 18:43:02.127951 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:02.128737 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24074
I20260929 18:43:02.134979 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24213
I20260929 18:43:02.142740 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24348
I20260929 18:43:02.149897 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 23961
I20260929 18:43:02.157260 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46441
--webserver_interface=127.11.118.254
--webserver_port=33201
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:46441
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:02.268543 24480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:02.268826 24480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:02.268865 24480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:02.273046 24480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:02.273144 24480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:02.273185 24480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:02.273227 24480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:02.273263 24480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:02.278898 24480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:46441
--ranger_config_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=33201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:02.279981 24480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:02.281009 24480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:02.289598 24488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:02.289598 24486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:02.289677 24485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:02.290102 24480 server_base.cc:1034] running on GCE node
I20260929 18:43:02.290810 24480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:02.292002 24480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:02.293200 24480 hybrid_clock.cc:648] HybridClock initialized: now 1790707382293183 us; error 30 us; skew 500 ppm
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:02.296084 24480 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:02.297005 24480 webserver.cc:460] Webserver started at http://127.11.118.254:33201/ using document root <none> and password file <none>
I20260929 18:43:02.297582 24480 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:02.297680 24480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:02.357419 24480 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:43:02.359916 24496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.361316 24480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:43:02.361470 24480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f70ba5e42cd1438695835e9f3ddc8c3b"
format_stamp: "Formatted at 2026-09-29 18:43:00 on dist-test-slave-cgv7"
server_key: "6b8bdd3af60cf8d7ba2cc19fb1a76cdf"
server_key_iv: "ec38e16b64a46ba550d318b438cb8945"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.362059 24480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:02.379387 24480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:02.382635 24480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:02.382870 24480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:02.390559 24480 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:46441
I20260929 18:43:02.390589 24548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:46441 every 8 connection(s)
I20260929 18:43:02.391705 24480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:02.395217 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24480
I20260929 18:43:02.395717 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:36851
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:46441
--webserver_port=41207
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:02.396445 24549 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:43:02.398186 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Bootstrap starting.
I20260929 18:43:02.415019 24549 log.cc:824] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:02.421841 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:02.422241 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Bootstrap complete.
I20260929 18:43:02.426434 24549 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:02.427100 24549 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70ba5e42cd1438695835e9f3ddc8c3b, State: Initialized, Role: FOLLOWER
I20260929 18:43:02.427641 24549 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:02.427793 24549 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:02.427891 24549 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:02.428006 24549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:43:02.429965 24549 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:02.430343 24549 leader_election.cc:302] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70ba5e42cd1438695835e9f3ddc8c3b; no voters:
I20260929 18:43:02.430721 24549 leader_election.cc:288] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 18:43:02.430828 24553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 2 FOLLOWER]: Leader election won for term 2
I20260929 18:43:02.431066 24553 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [term 2 LEADER]: Becoming Leader. State: Replica: f70ba5e42cd1438695835e9f3ddc8c3b, State: Running, Role: LEADER
I20260929 18:43:02.431434 24553 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } }
I20260929 18:43:02.432029 24549 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:02.432374 24553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70ba5e42cd1438695835e9f3ddc8c3b. Latest consensus state: current_term: 2 leader_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } } }
I20260929 18:43:02.432566 24553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:02.432734 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70ba5e42cd1438695835e9f3ddc8c3b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46441 } } }
I20260929 18:43:02.432839 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:02.433079 24557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:02.436364 24557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:02.436935 24549 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 18:43:02.437181 24557 catalog_manager.cc:1259] Loaded cluster ID: 40df884e6f364fd3a89cf9ac91c9bbe9
I20260929 18:43:02.437229 24557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:02.439600 24557 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:02.440240 24557 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Loaded TSK: 0
I20260929 18:43:02.441442 24557 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:43:02.543344 24551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:02.543757 24551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:02.543915 24551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:02.551159 24551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:02.551389 24551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:02.551545 24551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:02.560338 24551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:36851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=41207
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:02.561956 24551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:02.563340 24551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:02.576145 24589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:02.576498 24551 server_base.cc:1034] running on GCE node
W20260929 18:43:02.576835 24587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:02.579552 24586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:02.580438 24551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:02.581663 24551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:02.586485 24551 hybrid_clock.cc:648] HybridClock initialized: now 1790707382586437 us; error 63 us; skew 500 ppm
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:02.590526 24551 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:02.591840 24551 webserver.cc:460] Webserver started at http://127.11.118.193:41207/ using document root <none> and password file <none>
I20260929 18:43:02.592665 24551 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:02.592841 24551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:02.625172 24551 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260929 18:43:02.631275 24597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.633123 24551 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260929 18:43:02.633294 24551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "bcfd84d2f43f4c55b0b5e5633e042a91"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "f0676cf6b33e21024c6d64e27509f60f"
server_key_iv: "a362b74c07fde07a4a533841bc093de0"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.633927 24551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:02.652016 24551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:02.655730 24551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:02.655957 24551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:02.656692 24551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:02.657732 24551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:02.657809 24551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.657873 24551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:02.657913 24551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.669530 24551 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:36851
I20260929 18:43:02.669597 24710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:36851 every 8 connection(s)
I20260929 18:43:02.671005 24551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:02.676318 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24551
I20260929 18:43:02.676813 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:43059
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:46441
--webserver_port=38873
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:02.711782 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.673521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:47621 (local address 127.11.118.254:46441)
0929 18:43:02.674207 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:02.674229 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:02.674977 (+ 748us) server_negotiation.cc:407] Connection header received
0929 18:43:02.676162 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:02.676180 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:02.676666 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:02.677045 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:02.691007 (+ 13962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.692691 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.694958 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.695779 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.695949 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.704818 (+ 8869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:02.704826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:02.704844 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:02.704884 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:02.707070 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.707963 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.707970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.707975 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.708121 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.708512 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.708515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.708517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.708971 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:02.709097 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:02.710453 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0929 18:43:02.710758 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":125,"threads_started":1}
I20260929 18:43:02.712782 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.673652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.193:47621)
0929 18:43:02.674780 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:02.674836 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:02.675675 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:02.677411 (+ 1736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:02.677423 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:02.677792 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:02.690667 (+ 12875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.690698 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.692905 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.692910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.694769 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.694787 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.697630 (+ 2843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.697635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.698204 (+ 569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.702485 (+ 4281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:02.702512 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:02.704492 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:02.707408 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.707420 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.707435 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.707811 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.708226 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.708229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.708231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.708391 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.709150 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:02.709157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:02.709444 (+ 287us) client_negotiation.cc:769] Sending connection context
0929 18:43:02.709628 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 18:43:02.709960 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":132,"threads_started":1}
I20260929 18:43:02.715688 24711 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:02.715924 24711 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.716506 24711 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:02.718590 24711 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46441 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 18:43:02.718708 24711 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.718883 24711 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:02.719647 24711 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.719810 24711 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:02.720489 24711 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:43:02.819621 24715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:02.819907 24715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:02.819993 24715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:02.824390 24715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:02.824476 24715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:02.824574 24715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:02.830458 24715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:43059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=38873
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:02.831663 24715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:02.832827 24715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:02.842772 24720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:02.842792 24723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:02.843433 24715 server_base.cc:1034] running on GCE node
W20260929 18:43:02.846424 24721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:02.846863 24715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:02.847538 24715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:02.848717 24715 hybrid_clock.cc:648] HybridClock initialized: now 1790707382848703 us; error 30 us; skew 500 ppm
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:02.851804 24715 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:02.852926 24715 webserver.cc:460] Webserver started at http://127.11.118.194:38873/ using document root <none> and password file <none>
I20260929 18:43:02.853626 24715 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:02.853709 24715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:02.872442 24715 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:43:02.880563 24731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.882344 24715 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260929 18:43:02.882485 24715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "fba29dc0b67d467c8346eb1e75530a08"
format_stamp: "Formatted at 2026-09-29 18:43:01 on dist-test-slave-cgv7"
server_key: "c285d277eec76abcb5ef723c48bbec14"
server_key_iv: "b902c83245d0e5479aa39cbd035a2146"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:02.883087 24715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:02.903496 24715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:02.907146 24715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:02.907362 24715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:02.908123 24715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:02.909178 24715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:02.909248 24715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.909312 24715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:02.909348 24715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:02.921182 24715 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:43059
I20260929 18:43:02.922739 24715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:02.925505 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24715
I20260929 18:43:02.925973 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.118.212:39337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:35435
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:46441
--webserver_port=41463
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:02.924860 24844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:43059 every 8 connection(s)
Sep 29 18:43:02 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:02.948540 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.928674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:55171 (local address 127.11.118.254:46441)
0929 18:43:02.928821 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:02.928827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:02.929396 (+ 569us) server_negotiation.cc:407] Connection header received
0929 18:43:02.930684 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:02.930690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:02.930760 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:02.930843 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:02.932426 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.933359 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.935653 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.936080 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.936179 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.939771 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:02.939778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:02.939782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:02.939819 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:02.945890 (+ 6071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.946699 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.946704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.946708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.946799 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:02.947192 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:02.947195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:02.947197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:02.947431 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:02.947510 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:02.948213 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 18:43:02.948357 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:43:02.950556 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:02.927433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.194:55171)
0929 18:43:02.929197 (+ 1764us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:02.929252 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:02.930407 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:02.931294 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:02.931307 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:02.931685 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:02.932273 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.932289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.933856 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.933862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.935489 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:02.935505 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:02.936427 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:02.936433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:02.936667 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:02.937430 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:02.937458 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:02.939538 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:02.946089 (+ 6551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.946098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.946120 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.946577 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.946905 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:02.946910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:02.946913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:02.947090 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:02.947576 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:02.947582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:02.947932 (+ 350us) client_negotiation.cc:769] Sending connection context
0929 18:43:02.949099 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0929 18:43:02.949471 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1535,"thread_start_us":144,"threads_started":1}
I20260929 18:43:02.952188 24845 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:02.952432 24845 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.952980 24845 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:02.954662 24845 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46441 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 18:43:02.954777 24845 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.954937 24845 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:02.955731 24845 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:02.955879 24845 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:02.956600 24845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:43:03.075912 24847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:03.076351 24847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:03.076450 24847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:03.082134 24847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:03.082206 24847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:03.082321 24847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:03.088505 24847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.118.212:39337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:35435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=41463
--tserver_master_addrs=127.11.118.254:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:03.089783 24847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:03.091126 24847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:03.101248 24855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:03.101433 24854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:03.101230 24857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:03.103003 24847 server_base.cc:1034] running on GCE node
I20260929 18:43:03.103482 24847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:03.104355 24847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:03.105595 24847 hybrid_clock.cc:648] HybridClock initialized: now 1790707383105515 us; error 101 us; skew 500 ppm
Sep 29 18:43:03 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:03.109292 24847 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:03.110718 24847 webserver.cc:460] Webserver started at http://127.11.118.195:41463/ using document root <none> and password file <none>
I20260929 18:43:03.111792 24847 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:03.112027 24847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 18:43:03 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:03.141731 24847 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260929 18:43:03.145046 24865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:03.146720 24847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:03.146869 24847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "3d46661b65e44ea5b8f78537524c775d"
format_stamp: "Formatted at 2026-09-29 18:43:02 on dist-test-slave-cgv7"
server_key: "09a9c25a37331e44eb373f93ba332826"
server_key_iv: "8cdd97f6b5e993e84dc7617321409b8a"
server_key_version: "kuduclusterkey@0"
I20260929 18:43:03.147639 24847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:03.168116 24847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:03.172731 24847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:03.172956 24847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:03.173724 24847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:03.174860 24847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:03.174921 24847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:03.174975 24847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:03.175000 24847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:03.188129 24847 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:35435
I20260929 18:43:03.189458 24847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:03.191023 24978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:35435 every 8 connection(s)
I20260929 18:43:03.199333 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 24847
Sep 29 18:43:03 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:03.209503 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:03.191944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:36211 (local address 127.11.118.254:46441)
0929 18:43:03.192115 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:03.192121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:03.193170 (+ 1049us) server_negotiation.cc:407] Connection header received
0929 18:43:03.194460 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:03.194467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:03.194537 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:03.194638 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:03.196141 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.196949 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.198667 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.198881 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.199001 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:03.204741 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:03.204745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:03.204748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:03.204775 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:03.206784 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:03.207548 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:03.207551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:03.207553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:03.207640 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:03.208083 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:03.208086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:03.208088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:03.208279 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:03.208412 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:03.209165 (+ 753us) server_negotiation.cc:299] Negotiation successful
0929 18:43:03.209321 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":1792}
I20260929 18:43:03.210599 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:03.192430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46441 (local address 127.11.118.195:36211)
0929 18:43:03.193003 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:03.193064 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:03.194148 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:03.194864 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:03.194875 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:03.195284 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:03.195975 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:03.195997 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.197099 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.197105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:03.198542 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:03.198554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.199802 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.199822 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:03.200128 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:03.202738 (+ 2610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:03.202762 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:03.204541 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:03.206952 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:03.206961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:03.206985 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:03.207418 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:03.207755 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:03.207758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:03.207761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:03.207989 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:03.208403 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:03.208410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:03.208790 (+ 380us) client_negotiation.cc:769] Sending connection context
0929 18:43:03.209052 (+ 262us) client_negotiation.cc:240] Negotiation successful
0929 18:43:03.209418 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":134,"threads_started":1}
I20260929 18:43:03.211746 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:03.200409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.11.118.254:46441)
0929 18:43:03.201042 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:03.201050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:03.201062 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:43:03.201097 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:03.201100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:03.201143 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:03.201221 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:03.202194 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.203034 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.204341 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:03.204633 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:03.204755 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:03.206730 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:03.206736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:03.206740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:03.206779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:03.209150 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:03.209760 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:03.209763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:03.209765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:03.209849 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:03.210145 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:03.210148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:03.210150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:03.210399 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:03.210599 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:03.211419 (+ 820us) server_negotiation.cc:299] Negotiation successful
0929 18:43:03.211541 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"spinlock_wait_cycles":7936,"thread_start_us":97,"threads_started":1}
I20260929 18:43:03.211927 24979 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46441
I20260929 18:43:03.212183 24979 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:03.212745 24979 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:03.214462 24979 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46441 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 18:43:03.214596 24979 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:03.214779 24979 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:03.215569 24979 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:03.215741 24979 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
W20260929 18:43:03.216457 24979 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 18:43:03 dist-test-slave-cgv7 krb5kdc[23517](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:03.720732 24711 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:03.721063 24711 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:03.722450 24513 ts_manager.cc:194] Registered new tserver with Master: bcfd84d2f43f4c55b0b5e5633e042a91 (127.11.118.193:36851)
I20260929 18:43:03.724154 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:47621
I20260929 18:43:03.956830 24845 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:03.957150 24845 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:03.957846 24513 ts_manager.cc:194] Registered new tserver with Master: fba29dc0b67d467c8346eb1e75530a08 (127.11.118.194:43059)
I20260929 18:43:03.958732 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:55171
Sep 29 18:43:03 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:04.216723 24979 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:04.217079 24979 heartbeater.cc:511] Master 127.11.118.254:46441 requested a full tablet report, sending...
I20260929 18:43:04.217921 24513 ts_manager.cc:194] Registered new tserver with Master: 3d46661b65e44ea5b8f78537524c775d (127.11.118.195:35435)
I20260929 18:43:04.218621 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:36211
I20260929 18:43:04.223572 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:04.233402 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.225108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.11.118.254:46441)
0929 18:43:04.225341 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.225346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.225489 (+ 143us) server_negotiation.cc:407] Connection header received
0929 18:43:04.225645 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.225648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.225702 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.225798 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:04.226555 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.227098 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.227922 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.228170 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.228282 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.229698 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:04.229701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:04.229704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:04.229730 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:04.231319 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:04.231829 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:04.231835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:04.231836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:04.231926 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:04.232213 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:04.232216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:04.232217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:04.232395 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:04.232531 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.233086 (+ 555us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.233217 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260929 18:43:04.238059 24513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:43:04.242545 24513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:43:04.268285 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.259072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.11.118.193:36851)
0929 18:43:04.259337 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.259345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.261925 (+ 2580us) server_negotiation.cc:407] Connection header received
0929 18:43:04.262155 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.262160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.262399 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.262528 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:04.263042 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.264258 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.266010 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.266908 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.267003 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.267130 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.268000 (+ 870us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.268120 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260929 18:43:04.268677 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.259465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.11.118.194:43059)
0929 18:43:04.259797 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.259803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.262540 (+ 2737us) server_negotiation.cc:407] Connection header received
0929 18:43:04.262751 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.262756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.262929 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.263042 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:04.263441 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.264511 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.266612 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.267835 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.267943 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.268066 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.268421 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.268537 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":64,"threads_started":1}
I20260929 18:43:04.269248 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.258493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36851 (local address 127.0.0.1:44134)
0929 18:43:04.261827 (+ 3334us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.261840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:04.261993 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:04.262741 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:04.262746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:04.262767 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:04.262906 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.262914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.264417 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.264420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.265883 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.265893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.267292 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.267295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.267396 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.267827 (+ 431us) client_negotiation.cc:769] Sending connection context
0929 18:43:04.268938 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0929 18:43:04.269095 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3264,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260929 18:43:04.269989 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.258812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:43059 (local address 127.0.0.1:41746)
0929 18:43:04.262447 (+ 3635us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.262462 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:04.262604 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:04.263212 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:04.263215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:04.263231 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:04.263318 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.263324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.265340 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.265343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.266484 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.266494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.268157 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.268160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.268241 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.268260 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 18:43:04.269787 (+ 1527us) client_negotiation.cc:240] Negotiation successful
0929 18:43:04.269864 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3573,"thread_start_us":84,"threads_started":1}
I20260929 18:43:04.272732 24641 tablet_service.cc:1467] Processing CreateTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 (DEFAULT_TABLE table=test-table [id=acb64d579bdd48548b1d57f95c0dc2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:04.273267 24768 tablet_service.cc:1467] Processing CreateTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 (DEFAULT_TABLE table=test-table [id=acb64d579bdd48548b1d57f95c0dc2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:04.274380 24768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e36eb8e4ad4bf181df8e17ee9146a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:04.274923 24641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e36eb8e4ad4bf181df8e17ee9146a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:04.279455 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.257726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:35435 (local address 127.0.0.1:50312)
0929 18:43:04.261029 (+ 3303us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.261081 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:04.261208 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:04.270964 (+ 9756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:04.270967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:04.270988 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:04.271078 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.271085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.276750 (+ 5665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.276755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.277907 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.277920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.279043 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.279047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.279141 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.279170 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 18:43:04.279219 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 18:43:04.279302 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3129,"thread_start_us":129,"threads_started":1}
I20260929 18:43:04.280076 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.257865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.11.118.195:35435)
0929 18:43:04.261892 (+ 4027us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.261899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.261922 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:43:04.261999 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.262004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.270588 (+ 8584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.270789 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:04.275433 (+ 4644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.276596 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.278075 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.278896 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.279668 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.279802 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.279852 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.279941 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3923,"thread_start_us":64,"threads_started":1}
I20260929 18:43:04.283203 24913 tablet_service.cc:1467] Processing CreateTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 (DEFAULT_TABLE table=test-table [id=acb64d579bdd48548b1d57f95c0dc2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:04.284257 24913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e36eb8e4ad4bf181df8e17ee9146a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:04.295812 25015 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Bootstrap starting.
Sep 29 18:43:04 dist-test-slave-cgv7 krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for HTTP/127.11.118.212@KRBTEST.COM
I20260929 18:43:04.299396 25015 tablet_bootstrap.cc:654] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:04.300716 25015 log.cc:824] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:04.309630 25013 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Bootstrap starting.
I20260929 18:43:04.310828 25014 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Bootstrap starting.
I20260929 18:43:04.313653 25014 tablet_bootstrap.cc:654] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:04.314440 25013 tablet_bootstrap.cc:654] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:04.315440 25014 log.cc:824] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:04.316669 25015 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: No bootstrap required, opened a new log
I20260929 18:43:04.316996 25015 ts_tablet_manager.cc:1397] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260929 18:43:04.320061 25013 log.cc:824] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:04.321921 25015 raft_consensus.cc:348] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.322263 25015 raft_consensus.cc:374] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:04.322387 25015 raft_consensus.cc:729] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fba29dc0b67d467c8346eb1e75530a08, State: Initialized, Role: FOLLOWER
I20260929 18:43:04.323041 25013 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: No bootstrap required, opened a new log
I20260929 18:43:04.322965 25015 consensus_queue.cc:260] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.323271 25013 ts_tablet_manager.cc:1397] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.002s
I20260929 18:43:04.324271 25015 ts_tablet_manager.cc:1428] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 18:43:04.324723 24845 heartbeater.cc:503] Master 127.11.118.254:46441 was elected leader, sending a full tablet report...
I20260929 18:43:04.327497 25013 raft_consensus.cc:348] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.327896 25013 raft_consensus.cc:374] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:04.327982 25013 raft_consensus.cc:729] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfd84d2f43f4c55b0b5e5633e042a91, State: Initialized, Role: FOLLOWER
I20260929 18:43:04.328460 25013 consensus_queue.cc:260] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.329208 24711 heartbeater.cc:503] Master 127.11.118.254:46441 was elected leader, sending a full tablet report...
I20260929 18:43:04.329595 25013 ts_tablet_manager.cc:1428] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260929 18:43:04.332504 25014 tablet_bootstrap.cc:492] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: No bootstrap required, opened a new log
I20260929 18:43:04.332733 25014 ts_tablet_manager.cc:1397] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260929 18:43:04.337142 25014 raft_consensus.cc:348] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.338608 25014 raft_consensus.cc:374] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:04.338716 25014 raft_consensus.cc:729] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d46661b65e44ea5b8f78537524c775d, State: Initialized, Role: FOLLOWER
I20260929 18:43:04.339341 25014 consensus_queue.cc:260] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.354029 25014 ts_tablet_manager.cc:1428] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260929 18:43:04.354822 24979 heartbeater.cc:503] Master 127.11.118.254:46441 was elected leader, sending a full tablet report...
W20260929 18:43:04.423682 24712 tablet.cc:2369] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:43:04.430195 24846 tablet.cc:2369] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:43:04.443281 24980 tablet.cc:2369] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:43:04.490621 25019 raft_consensus.cc:482] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:04.490885 25019 raft_consensus.cc:504] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.492336 25019 leader_election.cc:288] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d46661b65e44ea5b8f78537524c775d (127.11.118.195:35435), bcfd84d2f43f4c55b0b5e5633e042a91 (127.11.118.193:36851)
I20260929 18:43:04.498495 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.493627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:47799 (local address 127.11.118.193:36851)
0929 18:43:04.494090 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.494095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.494109 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:04.494166 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.494169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.494213 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.494322 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:04.494690 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.495493 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.497141 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.497734 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.497826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.497875 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.498240 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.498352 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373}
I20260929 18:43:04.499455 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.493121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36851 (local address 127.11.118.194:47799)
0929 18:43:04.493845 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.493859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:04.493971 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:04.494355 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:04.494359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:04.494378 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:04.494532 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.494542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.495651 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.495654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.496984 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.496996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.497987 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.497991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.498102 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.498131 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 18:43:04.498210 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 18:43:04.499262 (+ 1052us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":961,"thread_start_us":88,"threads_started":1}
I20260929 18:43:04.500560 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e36eb8e4ad4bf181df8e17ee9146a2" candidate_uuid: "fba29dc0b67d467c8346eb1e75530a08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" is_pre_election: true
I20260929 18:43:04.500922 24647 raft_consensus.cc:2454] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fba29dc0b67d467c8346eb1e75530a08 in term 0.
I20260929 18:43:04.501470 24734 leader_election.cc:302] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcfd84d2f43f4c55b0b5e5633e042a91, fba29dc0b67d467c8346eb1e75530a08; no voters:
I20260929 18:43:04.501748 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.493551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:38379 (local address 127.11.118.195:35435)
0929 18:43:04.493679 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.493683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.493698 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:43:04.493754 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.493757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.493814 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.493901 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:04.497215 (+ 3314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.497934 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.499484 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.500066 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.500160 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.500206 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.501543 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.501631 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:43:04.501801 25019 raft_consensus.cc:2781] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:04.501927 25019 raft_consensus.cc:482] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:04.501979 25019 raft_consensus.cc:3037] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:04.503420 25019 raft_consensus.cc:504] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.503839 25019 leader_election.cc:288] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [CANDIDATE]: Term 1 election: Requested vote from peers 3d46661b65e44ea5b8f78537524c775d (127.11.118.195:35435), bcfd84d2f43f4c55b0b5e5633e042a91 (127.11.118.193:36851)
I20260929 18:43:04.504431 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e36eb8e4ad4bf181df8e17ee9146a2" candidate_uuid: "fba29dc0b67d467c8346eb1e75530a08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91"
I20260929 18:43:04.504566 24647 raft_consensus.cc:3037] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:04.505921 24647 raft_consensus.cc:2454] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fba29dc0b67d467c8346eb1e75530a08 in term 1.
I20260929 18:43:04.506443 24734 leader_election.cc:302] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcfd84d2f43f4c55b0b5e5633e042a91, fba29dc0b67d467c8346eb1e75530a08; no voters:
I20260929 18:43:04.506682 25019 raft_consensus.cc:2781] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:04.506925 25019 raft_consensus.cc:686] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 1 LEADER]: Becoming Leader. State: Replica: fba29dc0b67d467c8346eb1e75530a08, State: Running, Role: LEADER
I20260929 18:43:04.507321 25019 consensus_queue.cc:237] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } }
I20260929 18:43:04.507764 25023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.492710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:35435 (local address 127.11.118.194:38379)
0929 18:43:04.493060 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.493077 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:04.493182 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:04.495753 (+ 2571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:04.495755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:04.495774 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:04.497073 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.497080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.498109 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.498111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.499308 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:04.499318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.501278 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.501281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:04.501383 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.501405 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:43:04.501884 (+ 479us) client_negotiation.cc:240] Negotiation successful
0929 18:43:04.501965 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":1152,"thread_start_us":123,"threads_started":1}
I20260929 18:43:04.509187 24933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e36eb8e4ad4bf181df8e17ee9146a2" candidate_uuid: "fba29dc0b67d467c8346eb1e75530a08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d46661b65e44ea5b8f78537524c775d" is_pre_election: true
I20260929 18:43:04.509291 24932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e36eb8e4ad4bf181df8e17ee9146a2" candidate_uuid: "fba29dc0b67d467c8346eb1e75530a08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d46661b65e44ea5b8f78537524c775d"
I20260929 18:43:04.509528 24933 raft_consensus.cc:2454] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fba29dc0b67d467c8346eb1e75530a08 in term 0.
I20260929 18:43:04.512984 24513 catalog_manager.cc:5495] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 reported cstate change: term changed from 0 to 1, leader changed from <none> to fba29dc0b67d467c8346eb1e75530a08 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "fba29dc0b67d467c8346eb1e75530a08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fba29dc0b67d467c8346eb1e75530a08" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 43059 } health_report { overall_health: HEALTHY } } }
I20260929 18:43:04.567746 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.562519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.11.118.194:43059)
0929 18:43:04.562683 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:04.562689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:04.562776 (+ 87us) server_negotiation.cc:407] Connection header received
0929 18:43:04.562890 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:04.562896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:04.562993 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:04.563140 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:04.563737 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.564600 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.565899 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:04.566232 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:04.566395 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:04.566813 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:04.567277 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:04.567441 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:04.567486 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 18:43:04.567564 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 18:43:04.573750 24647 raft_consensus.cc:1261] T e4e36eb8e4ad4bf181df8e17ee9146a2 P bcfd84d2f43f4c55b0b5e5633e042a91 [term 1 FOLLOWER]: Refusing update from remote peer fba29dc0b67d467c8346eb1e75530a08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:04.573827 24932 raft_consensus.cc:3037] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:04.574621 25019 consensus_queue.cc:1035] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcfd84d2f43f4c55b0b5e5633e042a91" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:04.575366 24932 raft_consensus.cc:1261] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 1 FOLLOWER]: Refusing update from remote peer fba29dc0b67d467c8346eb1e75530a08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:04.575937 25019 consensus_queue.cc:1035] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d46661b65e44ea5b8f78537524c775d" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 35435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:04.583076 25030 mvcc.cc:205] Tried to move back new op lower bound from 7334737447206334464 to 7334737446952542208. Current Snapshot: MvccSnapshot[applied={T|T < 7334737447206334464}]
I20260929 18:43:04.585027 25035 mvcc.cc:205] Tried to move back new op lower bound from 7334737447206334464 to 7334737446952542208. Current Snapshot: MvccSnapshot[applied={T|T < 7334737447206334464}]
I20260929 18:43:04.589682 25033 mvcc.cc:205] Tried to move back new op lower bound from 7334737447206334464 to 7334737446952542208. Current Snapshot: MvccSnapshot[applied={T|T < 7334737447206334464}]
I20260929 18:43:04.595291 24512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:04.595496 24512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:04.598348 24512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f70ba5e42cd1438695835e9f3ddc8c3b: Sending DeleteTablet for 3 replicas of tablet e4e36eb8e4ad4bf181df8e17ee9146a2
I20260929 18:43:04.599313 24913 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:50312
I20260929 18:43:04.599323 24641 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:44134
I20260929 18:43:04.600133 24768 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e36eb8e4ad4bf181df8e17ee9146a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:41746
I20260929 18:43:04.600571 25047 tablet_replica.cc:330] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: stopping tablet replica
I20260929 18:43:04.600893 25047 raft_consensus.cc:2229] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:43:04.601033 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24551
I20260929 18:43:04.601246 25047 raft_consensus.cc:2258] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:04.602535 25046 tablet_replica.cc:330] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: stopping tablet replica
I20260929 18:43:04.602843 25046 raft_consensus.cc:2229] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:04.603300 25046 raft_consensus.cc:2258] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:04.607908 25046 ts_tablet_manager.cc:1905] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:04.610999 25046 ts_tablet_manager.cc:1918] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:04.611112 25046 log.cc:1196] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/e4e36eb8e4ad4bf181df8e17ee9146a2
I20260929 18:43:04.611471 25046 ts_tablet_manager.cc:1939] T e4e36eb8e4ad4bf181df8e17ee9146a2 P 3d46661b65e44ea5b8f78537524c775d: Deleting consensus metadata
W20260929 18:43:04.613462 24499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:36851 (error 108)
I20260929 18:43:04.613570 25047 ts_tablet_manager.cc:1905] T e4e36eb8e4ad4bf181df8e17ee9146a2 P fba29dc0b67d467c8346eb1e75530a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:04.613983 24499 catalog_manager.cc:4841] TS 3d46661b65e44ea5b8f78537524c775d (127.11.118.195:35435): tablet e4e36eb8e4ad4bf181df8e17ee9146a2 (table test-table [id=acb64d579bdd48548b1d57f95c0dc2a9]) successfully deleted
I20260929 18:43:04.614420 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24715
I20260929 18:43:04.617367 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.617041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36851 (local address 127.0.0.1:44148)
0929 18:43:04.617202 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.617277 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:36851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 18:43:04.617606 24499 catalog_manager.cc:4584] TS bcfd84d2f43f4c55b0b5e5633e042a91 (127.11.118.193:36851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4e36eb8e4ad4bf181df8e17ee9146a2: Network error: Client connection negotiation failed: client connection to 127.11.118.193:36851: connect: Connection refused (error 111)
I20260929 18:43:04.620918 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24847
I20260929 18:43:04.622125 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:04.621152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:43059 (local address 127.0.0.1:41762)
0929 18:43:04.621975 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:04.622036 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.194:43059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":734}
I20260929 18:43:04.627725 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 24480
I20260929 18:43:04.634950 11739 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 18:43:04.979960 11739 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 18:43:04.980060 11739 mini_ranger.cc:66] Stopping Ranger...
I20260929 18:43:05.325536 11739 mini_ranger.cc:68] Stopped Ranger
2026-09-29 18:43:05.325 UTC [23547] LOG: received smart shutdown request
2026-09-29 18:43:05.329 UTC [23547] LOG: background worker "logical replication launcher" (PID 23556) exited with exit code 1
2026-09-29 18:43:05.329 UTC [23551] LOG: shutting down
2026-09-29 18:43:05.513 UTC [23547] LOG: database system is shut down
2026-09-29T18:43:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50077 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:43:05 dist-test-slave-cgv7 krb5kdc[25057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:43:05 dist-test-slave-cgv7 krb5kdc[25057](info): set up 2 sockets
Sep 29 18:43:05 dist-test-slave-cgv7 krb5kdc[25057](info): commencing operation
krb5kdc: starting...
W20260929 18:43:06.678524 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:43:06 dist-test-slave-cgv7 krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:43:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:43:06.806624 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43703
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:06.919866 25070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:06.920172 25070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:06.920245 25070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:06.924218 25070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:06.924314 25070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:06.924350 25070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:06.924388 25070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:06.924428 25070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:06.929637 25070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:43703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:06.930802 25070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:06.931830 25070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:06.940274 25078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:06.940374 25075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:06.940368 25070 server_base.cc:1034] running on GCE node
W20260929 18:43:06.940292 25076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:06.941205 25070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:06.942396 25070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:06.943558 25070 hybrid_clock.cc:648] HybridClock initialized: now 1790707386943532 us; error 38 us; skew 500 ppm
Sep 29 18:43:06 dist-test-slave-cgv7 krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:06.946329 25070 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:06.947227 25070 webserver.cc:460] Webserver started at http://127.11.118.254:41801/ using document root <none> and password file <none>
I20260929 18:43:06.947791 25070 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:06.947893 25070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:06.948129 25070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:06.950212 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "f7437d6eeaeb4cd2812574548335a765"
format_stamp: "Formatted at 2026-09-29 18:43:06 on dist-test-slave-cgv7"
server_key: "f6617428d0ae40b7b256baadcd7a7dd7"
server_key_iv: "7eb32e8bfa889db265cdabff1065c1b8"
server_key_version: "encryptionkey@0"
I20260929 18:43:06.950834 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "f7437d6eeaeb4cd2812574548335a765"
format_stamp: "Formatted at 2026-09-29 18:43:06 on dist-test-slave-cgv7"
server_key: "f6617428d0ae40b7b256baadcd7a7dd7"
server_key_iv: "7eb32e8bfa889db265cdabff1065c1b8"
server_key_version: "encryptionkey@0"
I20260929 18:43:06.954566 25070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:43:06.957332 25085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:06.959018 25070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:06.959199 25070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f7437d6eeaeb4cd2812574548335a765"
format_stamp: "Formatted at 2026-09-29 18:43:06 on dist-test-slave-cgv7"
server_key: "f6617428d0ae40b7b256baadcd7a7dd7"
server_key_iv: "7eb32e8bfa889db265cdabff1065c1b8"
server_key_version: "encryptionkey@0"
I20260929 18:43:06.959334 25070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:06.986960 25070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:06.989987 25070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:06.990150 25070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:06.996884 25070 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:43703
I20260929 18:43:06.997186 25137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:43703 every 8 connection(s)
I20260929 18:43:06.997901 25070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:07.000761 25138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:07.003510 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25070
I20260929 18:43:07.003649 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:43:07.003935 11739 external_mini_cluster.cc:1428] Setting key dc4b5e02fa846a9d987c9087e75057fd
I20260929 18:43:07.005259 25138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.118.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.253@KRBTEST.COM" created.
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:07.022833 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.008917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.11.118.254:43703)
0929 18:43:07.009285 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.009300 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.009351 (+ 51us) server_negotiation.cc:407] Connection header received
0929 18:43:07.009859 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.009869 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.010127 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.010399 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.011260 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.012673 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.013716 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.014258 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.014452 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.017277 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.017283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.017295 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.017317 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.019071 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.019696 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.019701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.019703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.019821 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.020456 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.020459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.020460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.020908 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.021027 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.021525 (+ 498us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.021934 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":181,"server-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
I20260929 18:43:07.022837 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.007968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43703 (local address 127.0.0.1:53010)
0929 18:43:07.008484 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.008538 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.009504 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.010429 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.010433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.010786 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.011088 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.011103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.012828 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.012836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.013604 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.013614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.014483 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.014487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.014626 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.015437 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.015455 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.017118 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.019192 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.019204 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.019219 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.019552 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.020103 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.020109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.020114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.020309 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.021057 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.021067 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.021250 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.021460 (+ 210us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.021866 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":150,"threads_started":1}
I20260929 18:43:07.025480 25138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:43:07.026333 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.026001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39654)
0929 18:43:07.026140 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.026213 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 18:43:07.026597 25088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
I20260929 18:43:07.027591 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.027394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39664)
0929 18:43:07.027495 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.027540 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 18:43:07.027798 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:44135: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
I20260929 18:43:07.033928 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:44135
--webserver_interface=127.11.118.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:07.077075 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } attempt: 1
I20260929 18:43:07.078086 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.077698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39666)
0929 18:43:07.077944 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.078009 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 18:43:07.078830 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.078635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39668)
0929 18:43:07.078734 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.078777 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 18:43:07.079001 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:44135: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
I20260929 18:43:07.147168 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } attempt: 2
W20260929 18:43:07.147289 25145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:07.147701 25145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:07.147786 25145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 18:43:07.148110 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.147799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39674)
0929 18:43:07.147972 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.148034 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 18:43:07.148862 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.148671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39680)
0929 18:43:07.148767 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.148812 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 18:43:07.149071 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:44135: Network error: Client connection negotiation failed: client connection to 127.11.118.253:44135: connect: Connection refused (error 111)
W20260929 18:43:07.152328 25145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:07.152406 25145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:07.152452 25145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:07.152488 25145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:07.152505 25145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:07.158252 25145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:44135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.118.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:07.159498 25145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:07.160554 25145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:07.169229 25151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:07.169246 25153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:07.169291 25150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:07.169653 25145 server_base.cc:1034] running on GCE node
I20260929 18:43:07.170174 25145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:07.171347 25145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:07.172521 25145 hybrid_clock.cc:648] HybridClock initialized: now 1790707387172498 us; error 36 us; skew 500 ppm
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:07.175455 25145 init.cc:377] Logged in from keytab as kudu/127.11.118.253@KRBTEST.COM (short username kudu)
I20260929 18:43:07.176364 25145 webserver.cc:460] Webserver started at http://127.11.118.253:37433/ using document root <none> and password file <none>
I20260929 18:43:07.176894 25145 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:07.176954 25145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:07.177119 25145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:07.179251 25145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data/instance:
uuid: "14f3c2439bb346268063430f331f553e"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "2c4e4eee6361a1cc992a1342c24a540b"
server_key_iv: "37a78e0aec1dead045c8a4450478a2a0"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.179765 25145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal/instance:
uuid: "14f3c2439bb346268063430f331f553e"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "2c4e4eee6361a1cc992a1342c24a540b"
server_key_iv: "37a78e0aec1dead045c8a4450478a2a0"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.183568 25145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:07.186045 25160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:07.187414 25145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:07.187520 25145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "14f3c2439bb346268063430f331f553e"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "2c4e4eee6361a1cc992a1342c24a540b"
server_key_iv: "37a78e0aec1dead045c8a4450478a2a0"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.187616 25145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:07.205348 25145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:07.208879 25145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:07.209048 25145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:07.216481 25145 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.253:44135
I20260929 18:43:07.216800 25212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.253:44135 every 8 connection(s)
I20260929 18:43:07.217516 25145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
I20260929 18:43:07.220338 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25145
I20260929 18:43:07.220520 25213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:07.220593 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-1/wal/instance
I20260929 18:43:07.220912 11739 external_mini_cluster.cc:1428] Setting key 066464c4494b8be6b3003968e8607e21
I20260929 18:43:07.225124 25213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.118.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.252@KRBTEST.COM" created.
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:07.240271 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } attempt: 3
I20260929 18:43:07.240326 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.227726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.11.118.254:43703)
0929 18:43:07.227923 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.227930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.228792 (+ 862us) server_negotiation.cc:407] Connection header received
0929 18:43:07.229811 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.229814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.229862 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.229954 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.231348 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.231864 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.233288 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.233552 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.233676 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.236578 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.236581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.236583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.236609 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.238101 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.238674 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.238679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.238681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.238770 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.239130 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.239133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.239135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.239333 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.239442 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.239986 (+ 544us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.240135 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 18:43:07.241501 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.228133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43703 (local address 127.0.0.1:53018)
0929 18:43:07.228610 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.228663 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.229604 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.230164 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.230204 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.230648 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.231180 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.231195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.232022 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.232026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.233146 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.233156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.233723 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.233727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.233942 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.234668 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.234686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.236376 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.238242 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.238248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.238288 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.238568 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.238874 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.238877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.238879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.239027 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.239441 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.239450 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.239722 (+ 272us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.239915 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.240297 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":135,"threads_started":1}
I20260929 18:43:07.242995 25213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
I20260929 18:43:07.250377 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:38407
--webserver_interface=127.11.118.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:07.252218 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.240758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39696)
0929 18:43:07.240896 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.240915 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.240995 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.242044 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.242047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.242237 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.242404 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.242414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.243602 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.243606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.244302 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.244312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.244998 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.245002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.245107 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.245670 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.245688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.247412 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.249778 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.249782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.249785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.250103 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.250526 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.250530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.250531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.250637 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.251581 (+ 944us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.251587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.251713 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.251841 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.252035 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 18:43:07.252269 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.241094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.11.118.253:44135)
0929 18:43:07.241576 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.241586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.241606 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:43:07.241665 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.241672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.241892 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.242049 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.242561 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.243456 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.244490 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.244882 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.244965 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.247562 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.247572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.247607 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.247639 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.249654 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.250242 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.250248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.250250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.250406 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.251050 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.251057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.251061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.251425 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.251596 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.251951 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.252114 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"spinlock_wait_cycles":2816,"thread_start_us":174,"threads_started":1}
I20260929 18:43:07.253861 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.243533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39712)
0929 18:43:07.243665 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.243680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.243750 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.244146 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.244148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.244324 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.244404 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.244409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.245284 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.245307 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.245943 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.245952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.246610 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.246614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.246714 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.247368 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.247379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.248864 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.251886 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.251888 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.251891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.252478 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.252799 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.252802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.252805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.252900 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.253372 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.253375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.253470 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.253581 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.253707 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"spinlock_wait_cycles":2304}
I20260929 18:43:07.254076 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.243634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.11.118.253:44135)
0929 18:43:07.243967 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.243973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.243982 (+ 9us) server_negotiation.cc:407] Connection header received
0929 18:43:07.244006 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.244008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.244038 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.244136 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.244529 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.245162 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.246055 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.246487 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.246575 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.249024 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.249026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.249029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.249052 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.251781 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.252587 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.252592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.252594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.252695 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.252997 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.253002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.253005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.253254 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.253357 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.253664 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.253797 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":128,"threads_started":1}
I20260929 18:43:07.254552 25138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:43:07.255039 25213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:43:07.255280 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.255024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36874)
0929 18:43:07.255144 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.255203 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 18:43:07.255622 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.255409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36878)
0929 18:43:07.255495 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.255564 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 18:43:07.256067 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.255816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36894)
0929 18:43:07.255959 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.256007 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260929 18:43:07.256073 25163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
W20260929 18:43:07.256561 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
I20260929 18:43:07.257009 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.256774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36908)
0929 18:43:07.256919 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.256961 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260929 18:43:07.257170 25213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
I20260929 18:43:07.287753 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 1
I20260929 18:43:07.288719 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.288409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36920)
0929 18:43:07.288574 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.288638 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 18:43:07.289489 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.289262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36924)
0929 18:43:07.289377 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.289427 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 18:43:07.289620 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
I20260929 18:43:07.306439 25213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 1
I20260929 18:43:07.307428 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.307123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36928)
0929 18:43:07.307283 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.307346 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 18:43:07.308199 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.307964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36944)
0929 18:43:07.308090 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.308141 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 18:43:07.308338 25213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
I20260929 18:43:07.364802 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 2
I20260929 18:43:07.365759 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.365417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36950)
0929 18:43:07.365591 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.365663 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 18:43:07.366611 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.366382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36964)
0929 18:43:07.366493 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.366547 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 18:43:07.366879 25138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
W20260929 18:43:07.368047 25220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:07.368312 25220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:07.368397 25220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:07.372733 25220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:07.372808 25220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:07.372828 25220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:07.372846 25220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:07.372896 25220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:07.376513 25213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 2
I20260929 18:43:07.377338 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.377052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36976)
0929 18:43:07.377191 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.377259 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:43:07.378074 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.377848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:36984)
0929 18:43:07.377964 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.378019 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 18:43:07.378202 25213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:38407: Network error: Client connection negotiation failed: client connection to 127.11.118.252:38407: connect: Connection refused (error 111)
I20260929 18:43:07.378978 25220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:43703,127.11.118.253:44135,127.11.118.252:38407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:38407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.118.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:07.380113 25220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:07.381143 25220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:07.389858 25227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:07.389959 25229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:07.389858 25226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:07.390529 25220 server_base.cc:1034] running on GCE node
I20260929 18:43:07.391149 25220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:07.392375 25220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:07.393589 25220 hybrid_clock.cc:648] HybridClock initialized: now 1790707387393563 us; error 39 us; skew 500 ppm
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:07.396564 25220 init.cc:377] Logged in from keytab as kudu/127.11.118.252@KRBTEST.COM (short username kudu)
I20260929 18:43:07.397526 25220 webserver.cc:460] Webserver started at http://127.11.118.252:43253/ using document root <none> and password file <none>
I20260929 18:43:07.398167 25220 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:07.398231 25220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:07.398486 25220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:07.400686 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data/instance:
uuid: "806bdee3c8d045f9910d8578c2f549f3"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "a9aa83c871b13a7d2e382addbcd74f01"
server_key_iv: "60222a598e2f233998254f8165808374"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.401314 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal/instance:
uuid: "806bdee3c8d045f9910d8578c2f549f3"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "a9aa83c871b13a7d2e382addbcd74f01"
server_key_iv: "60222a598e2f233998254f8165808374"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.405282 25220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:43:07.407845 25236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:07.409222 25220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:43:07.409420 25220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "806bdee3c8d045f9910d8578c2f549f3"
format_stamp: "Formatted at 2026-09-29 18:43:07 on dist-test-slave-cgv7"
server_key: "a9aa83c871b13a7d2e382addbcd74f01"
server_key_iv: "60222a598e2f233998254f8165808374"
server_key_version: "encryptionkey@0"
I20260929 18:43:07.409550 25220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:07.421475 25220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:07.425140 25220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:07.425382 25220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:07.433319 25220 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.252:38407
I20260929 18:43:07.433331 25288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.252:38407 every 8 connection(s)
I20260929 18:43:07.434412 25220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
I20260929 18:43:07.437397 25289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:07.438745 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25220
I20260929 18:43:07.438889 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790707223158922-11739-0/minicluster-data/master-2/wal/instance
I20260929 18:43:07.439194 11739 external_mini_cluster.cc:1428] Setting key 8380a9e25b9b1057041200f796fd652b
I20260929 18:43:07.441946 25289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:07.455215 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.444214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.11.118.254:43703)
0929 18:43:07.444379 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.444383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.444732 (+ 349us) server_negotiation.cc:407] Connection header received
0929 18:43:07.444949 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.444951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.445002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.445094 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.445965 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.446731 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.447896 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.448128 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.448219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.450896 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.450899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.450901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.450927 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.453143 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.453789 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.453793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.453797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.453921 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.454240 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.454244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.454245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.454513 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.454603 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.454988 (+ 385us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.455088 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:43:07.455619 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.440644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36996 (local address 127.11.118.252:38407)
0929 18:43:07.441091 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.441106 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.441144 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:43:07.441800 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.441817 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.442146 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.442537 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.443479 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.445129 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.446258 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.447049 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.447181 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.450215 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.450228 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.450240 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.450262 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.451998 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.452443 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.452453 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.452458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.452559 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.452851 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.452854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.452856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.453270 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.453485 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.454092 (+ 607us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.454506 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260929 18:43:07.455619 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.444140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:43703 (local address 127.0.0.1:53024)
0929 18:43:07.444591 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.444640 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.444793 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.445148 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.445153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.445661 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.445813 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.445820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.446868 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.446870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.447742 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.447756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.448311 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.448317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.448472 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.449004 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.449029 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.450669 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.453291 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.453300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.453309 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.453615 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.454030 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.454033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.454035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.454119 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.454626 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.454632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.454747 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.454907 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.455091 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":184,"threads_started":1}
I20260929 18:43:07.456921 25289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:07.465752 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.457444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:44135 (local address 127.0.0.1:39718)
0929 18:43:07.457604 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.457618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.457695 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.457967 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.457970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.458166 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.458313 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.458322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.459060 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.459067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.459590 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.459597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.460065 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.460068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.460169 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.460630 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.460643 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.462102 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.464220 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.464225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.464228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.464419 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.464714 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.464721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.464723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.464803 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.465188 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.465192 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.465314 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.465428 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.465563 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260929 18:43:07.465766 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.457535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.11.118.253:44135)
0929 18:43:07.457684 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.457687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.457700 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:43:07.457806 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.457810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.457861 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.457957 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.458421 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.458950 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.459706 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.459954 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.460067 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.462250 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.462255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.462258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.462304 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.464081 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.464522 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.464527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.464529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.464611 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.464897 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.464902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.464903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.465084 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.465188 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.465476 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.465575 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 18:43:07.466100 25138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 3
I20260929 18:43:07.466759 25289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:43:07.467308 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.458185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.11.118.254:43703)
0929 18:43:07.458326 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.458330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.458344 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:07.458470 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.458473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.458514 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.458601 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.459183 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.459671 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.460480 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.460839 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.461068 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.463771 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.463775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.463778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.463804 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.465212 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.465670 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.465674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.465676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.465753 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.466109 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.466113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.466114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.466352 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.466582 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.466983 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.467132 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:43:07.469358 25213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } attempt: 3
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Sep 29 18:43:07 dist-test-slave-cgv7 krb5kdc[25057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:43:07.478682 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.467207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:37020)
0929 18:43:07.467329 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.467345 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.467430 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.467941 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.467943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.468102 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.468211 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.468218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.469111 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.469115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.469973 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.469982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.471082 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.471085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.471220 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.471653 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.471667 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.473665 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.477191 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.477199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.477392 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.477658 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.477665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.477745 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.478167 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.478170 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.478253 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.478354 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.478508 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":25}
I20260929 18:43:07.478705 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.466515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:37010)
0929 18:43:07.466632 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.466646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.466714 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.466984 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.466987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.467187 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.467298 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.467305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.468995 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.468998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.469726 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.469736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.470375 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.470379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.470462 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.471751 (+ 1289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.471762 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.474082 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.476119 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.476123 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.476125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.476337 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.477513 (+ 1176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.477519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.477619 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.478177 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.478181 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.478336 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.478427 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.478563 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 18:43:07.479043 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.467428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.11.118.252:38407)
0929 18:43:07.467739 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.467747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.467758 (+ 11us) server_negotiation.cc:407] Connection header received
0929 18:43:07.467799 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.467805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.467846 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.467933 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.468336 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.469032 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.470302 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.470964 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.471062 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.473872 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.473878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.473884 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.473913 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.477091 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477486 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.477493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.477495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.477571 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477845 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.477851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.477853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.478059 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.478488 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.478560 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.478672 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":123,"threads_started":1}
I20260929 18:43:07.480062 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.466598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.11.118.252:38407)
0929 18:43:07.466698 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.466702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.466715 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:43:07.466806 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.466809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.466850 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.467511 (+ 661us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.467616 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.468774 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.469866 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.470216 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.470678 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.474215 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.474219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.474221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.474243 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.476007 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.476874 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.476879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.476881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.476976 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.477744 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.477750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.477752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.478055 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.479587 (+ 1532us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.479659 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.479900 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":2432}
I20260929 18:43:07.480069 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.468752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.11.118.253:44135)
0929 18:43:07.468868 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.468872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.468952 (+ 80us) server_negotiation.cc:407] Connection header received
0929 18:43:07.469063 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.469066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.469112 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.469178 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.469962 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.470716 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.472221 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.472607 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.472711 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.476178 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.476181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.476184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.476205 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.477908 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.478414 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.478418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.478419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.478500 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.478797 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.478801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.478803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.479012 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.479217 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.479822 (+ 605us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.479928 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":41}
I20260929 18:43:07.480901 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.469668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:38407 (local address 127.0.0.1:37036)
0929 18:43:07.469893 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:07.469906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:07.469965 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:07.470668 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:07.470671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:07.470879 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:07.470966 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.470971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.472155 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.472161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.472829 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:07.472838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.473426 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.473430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:07.473526 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.473974 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:07.473991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:07.475982 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:07.479281 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.479284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.479287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.479537 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.479850 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:07.479855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:07.479860 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:07.479952 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:07.480404 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:07.480407 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:07.480493 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 18:43:07.480579 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:43:07.480714 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":2}
I20260929 18:43:07.481814 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.470099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.11.118.252:38407)
0929 18:43:07.470454 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.470461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.470473 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:43:07.470505 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.470508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.470543 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.471082 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.471239 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.472017 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.472938 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.473300 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.473462 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.476141 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.476146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.476149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.476173 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.479170 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.479648 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.479650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.479653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.479726 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.480063 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.480068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.480072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.480294 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.481482 (+ 1188us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.481555 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.481654 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":293,"thread_start_us":110,"threads_started":1}
I20260929 18:43:07.482504 25289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: Bootstrap starting.
I20260929 18:43:07.483271 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:43:07.483489 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: Bootstrap starting.
I20260929 18:43:07.484997 25213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: Bootstrap starting.
I20260929 18:43:07.485379 25289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:07.485944 25138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:07.486029 25289 log.cc:824] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:07.486766 25138 log.cc:824] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:07.487551 25213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:07.488405 25213 log.cc:824] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:07.489519 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: No bootstrap required, opened a new log
I20260929 18:43:07.491762 25213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: No bootstrap required, opened a new log
I20260929 18:43:07.495443 25138 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.495684 25138 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:07.495743 25138 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7437d6eeaeb4cd2812574548335a765, State: Initialized, Role: FOLLOWER
I20260929 18:43:07.496078 25213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.496274 25213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:07.496325 25213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f3c2439bb346268063430f331f553e, State: Initialized, Role: FOLLOWER
I20260929 18:43:07.496253 25138 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.496811 25213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.497082 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.486778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.11.118.254:43703)
0929 18:43:07.486906 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.486911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.487290 (+ 379us) server_negotiation.cc:407] Connection header received
0929 18:43:07.487818 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.487822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.487865 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.487965 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.488952 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.489742 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.490886 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.491200 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.491309 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.493400 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.493406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.493409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.493434 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.495203 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.495701 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.495705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.495707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.495790 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.496101 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.496105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.496107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.496340 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.496446 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.496808 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.496923 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:43:07.497684 25138 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:07.497933 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.487227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.11.118.253:44135)
0929 18:43:07.487387 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.487390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.487932 (+ 542us) server_negotiation.cc:407] Connection header received
0929 18:43:07.488021 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.488024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.488071 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.488139 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.488943 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.489632 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.490923 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.491254 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.491356 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.493310 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.493313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.493316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.493341 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.495065 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.495548 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.495551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.495553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.495636 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.496130 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.496134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.496135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.496341 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.496458 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.497076 (+ 618us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.497737 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 18:43:07.498225 25213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:07.500145 25306 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.500309 25306 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.501190 25307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.502418 25307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.504693 25289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: No bootstrap required, opened a new log
I20260929 18:43:07.506626 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:07.487518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.11.118.252:38407)
0929 18:43:07.487619 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:07.487623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:07.497554 (+ 9931us) server_negotiation.cc:407] Connection header received
0929 18:43:07.497691 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:07.497695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:07.497749 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:07.497831 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:07.498751 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.499578 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.500668 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:07.501032 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:07.501116 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:07.502950 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:07.502954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:07.502958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:07.502983 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:07.504713 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.505192 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.505196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.505198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.505284 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:07.505608 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:07.505612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:07.505613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:07.505820 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:07.505970 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:07.506360 (+ 390us) server_negotiation.cc:299] Negotiation successful
0929 18:43:07.506481 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 18:43:07.506217 25070 master_runner.cc:186] Error getting master registration for 127.11.118.252:38407: OK, instance_id { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" instance_seqno: 1790707387430793 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:43:07.508993 25070 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:43:07.509543 25289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.509786 25289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:07.509850 25289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806bdee3c8d045f9910d8578c2f549f3, State: Initialized, Role: FOLLOWER
I20260929 18:43:07.510362 25289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
W20260929 18:43:07.511019 25323 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:43:07.511127 25323 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:43:07.511804 25289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:07.509708 25145 master_runner.cc:186] Error getting master registration for 127.11.118.252:38407: OK, instance_id { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" instance_seqno: 1790707387430793 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:43:07.513190 25145 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 18:43:07.514868 25329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:43:07.514989 25329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260929 18:43:07.516960 25343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:43:07.517038 25343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:43:07.517329 25331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.517452 25331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.520797 25220 master_runner.cc:263] No leader master found from master 806bdee3c8d045f9910d8578c2f549f3
I20260929 18:43:07.520884 25220 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:43:07.562258 25306 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:07.562526 25306 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.563706 25306 leader_election.cc:288] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14f3c2439bb346268063430f331f553e (127.11.118.253:44135), 806bdee3c8d045f9910d8578c2f549f3 (127.11.118.252:38407)
I20260929 18:43:07.564503 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7437d6eeaeb4cd2812574548335a765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f3c2439bb346268063430f331f553e" is_pre_election: true
I20260929 18:43:07.564842 25187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7437d6eeaeb4cd2812574548335a765 in term 0.
I20260929 18:43:07.565274 25088 leader_election.cc:302] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f3c2439bb346268063430f331f553e, f7437d6eeaeb4cd2812574548335a765; no voters:
I20260929 18:43:07.565605 25306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:07.565686 25306 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:07.565734 25306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:07.567102 25306 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.567454 25306 leader_election.cc:288] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [CANDIDATE]: Term 1 election: Requested vote from peers 14f3c2439bb346268063430f331f553e (127.11.118.253:44135), 806bdee3c8d045f9910d8578c2f549f3 (127.11.118.252:38407)
I20260929 18:43:07.567904 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7437d6eeaeb4cd2812574548335a765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f3c2439bb346268063430f331f553e"
I20260929 18:43:07.568029 25187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:07.569320 25187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7437d6eeaeb4cd2812574548335a765 in term 1.
I20260929 18:43:07.569653 25088 leader_election.cc:302] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f3c2439bb346268063430f331f553e, f7437d6eeaeb4cd2812574548335a765; no voters:
I20260929 18:43:07.570393 25263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7437d6eeaeb4cd2812574548335a765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806bdee3c8d045f9910d8578c2f549f3" is_pre_election: true
I20260929 18:43:07.570744 25263 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7437d6eeaeb4cd2812574548335a765 in term 0.
I20260929 18:43:07.571038 25262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7437d6eeaeb4cd2812574548335a765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806bdee3c8d045f9910d8578c2f549f3"
I20260929 18:43:07.571153 25262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:07.572377 25262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7437d6eeaeb4cd2812574548335a765 in term 1.
I20260929 18:43:07.573092 25306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:07.573349 25306 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [term 1 LEADER]: Becoming Leader. State: Replica: f7437d6eeaeb4cd2812574548335a765, State: Running, Role: LEADER
I20260929 18:43:07.573709 25306 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } }
I20260929 18:43:07.578972 25306 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7437d6eeaeb4cd2812574548335a765. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.579123 25306 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:07.579495 25347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:07.580426 25347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:07.588223 25262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [term 1 FOLLOWER]: Refusing update from remote peer f7437d6eeaeb4cd2812574548335a765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:07.590615 25345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:07.594686 25331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7437d6eeaeb4cd2812574548335a765. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.594822 25331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.595825 25187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [term 1 FOLLOWER]: Refusing update from remote peer f7437d6eeaeb4cd2812574548335a765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:07.596364 25345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:07.600586 25307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7437d6eeaeb4cd2812574548335a765. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.600728 25307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.609347 25345 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.609519 25345 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:07.610759 25307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.610903 25307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.611284 25348 mvcc.cc:205] Tried to move back new op lower bound from 7334737459551174656 to 7334737459514867712. Current Snapshot: MvccSnapshot[applied={T|T < 7334737459551174656}]
I20260929 18:43:07.612134 25351 mvcc.cc:205] Tried to move back new op lower bound from 7334737459551174656 to 7334737459514867712. Current Snapshot: MvccSnapshot[applied={T|T < 7334737459551174656}]
I20260929 18:43:07.614475 25306 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.614598 25306 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:07.615857 25331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7437d6eeaeb4cd2812574548335a765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7437d6eeaeb4cd2812574548335a765" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 43703 } } peers { permanent_uuid: "14f3c2439bb346268063430f331f553e" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 44135 } } peers { permanent_uuid: "806bdee3c8d045f9910d8578c2f549f3" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 38407 } } }
I20260929 18:43:07.615994 25331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:43:07.616493 25349 mvcc.cc:205] Tried to move back new op lower bound from 7334737459551174656 to 7334737459514867712. Current Snapshot: MvccSnapshot[applied={T|T < 7334737459551174656}]
I20260929 18:43:07.621002 25347 catalog_manager.cc:1347] Generated new cluster ID: c96f5ab956124b6882174bbc7f26fee5
I20260929 18:43:07.621089 25347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:07.640170 25347 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:43:07.641537 25347 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:07.663228 25347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7437d6eeaeb4cd2812574548335a765: Generated new TSK 0
I20260929 18:43:07.664172 25347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:43:08.516346 25329 catalog_manager.cc:1259] Loaded cluster ID: c96f5ab956124b6882174bbc7f26fee5
I20260929 18:43:08.516438 25329 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: loading cluster ID for follower catalog manager: success
I20260929 18:43:08.518250 25343 catalog_manager.cc:1259] Loaded cluster ID: c96f5ab956124b6882174bbc7f26fee5
I20260929 18:43:08.518355 25343 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: loading cluster ID for follower catalog manager: success
I20260929 18:43:08.518683 25329 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: acquiring CA information for follower catalog manager: success
I20260929 18:43:08.520078 25329 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14f3c2439bb346268063430f331f553e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:43:08.520388 25343 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: acquiring CA information for follower catalog manager: success
I20260929 18:43:08.521395 25343 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 806bdee3c8d045f9910d8578c2f549f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:43:08.737828 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25070
I20260929 18:43:08.744436 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25145
I20260929 18:43:08.750517 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25220
2026-09-29T18:43:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3121 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:43:08 dist-test-slave-cgv7 krb5kdc[25366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:43:08 dist-test-slave-cgv7 krb5kdc[25366](info): set up 2 sockets
Sep 29 18:43:08 dist-test-slave-cgv7 krb5kdc[25366](info): commencing operation
krb5kdc: starting...
W20260929 18:43:09.798591 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:43:09 dist-test-slave-cgv7 krb5kdc[25366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:43:09.929790 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42221
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:10.044235 25379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:10.044586 25379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:10.044673 25379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:10.048941 25379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:10.049038 25379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:10.049077 25379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:10.049132 25379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:10.049170 25379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:10.054620 25379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:10.055850 25379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:10.056918 25379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:10.065898 25387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:10.065905 25384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:10.065905 25385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:10.067287 25379 server_base.cc:1034] running on GCE node
I20260929 18:43:10.067921 25379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:10.069105 25379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:10.070320 25379 hybrid_clock.cc:648] HybridClock initialized: now 1790707390070258 us; error 31 us; skew 500 ppm
Sep 29 18:43:10 dist-test-slave-cgv7 krb5kdc[25366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:10.073513 25379 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:10.074690 25379 webserver.cc:460] Webserver started at http://127.11.118.254:46877/ using document root <none> and password file <none>
I20260929 18:43:10.075309 25379 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:10.075379 25379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:10.075630 25379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:10.077809 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "aeb765236f0e4212948bf442aac59832"
format_stamp: "Formatted at 2026-09-29 18:43:10 on dist-test-slave-cgv7"
server_key: "eef2b503cf6eaeb3b198c72cc2fa8f93"
server_key_iv: "c50ca97ae63751cb77e5bcc75dfd0d7c"
server_key_version: "encryptionkey@0"
I20260929 18:43:10.078437 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "aeb765236f0e4212948bf442aac59832"
format_stamp: "Formatted at 2026-09-29 18:43:10 on dist-test-slave-cgv7"
server_key: "eef2b503cf6eaeb3b198c72cc2fa8f93"
server_key_iv: "c50ca97ae63751cb77e5bcc75dfd0d7c"
server_key_version: "encryptionkey@0"
I20260929 18:43:10.082628 25379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:10.085435 25394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:10.086982 25379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:10.087093 25379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "aeb765236f0e4212948bf442aac59832"
format_stamp: "Formatted at 2026-09-29 18:43:10 on dist-test-slave-cgv7"
server_key: "eef2b503cf6eaeb3b198c72cc2fa8f93"
server_key_iv: "c50ca97ae63751cb77e5bcc75dfd0d7c"
server_key_version: "encryptionkey@0"
I20260929 18:43:10.087195 25379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:10.115638 25379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:10.119158 25379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:10.119330 25379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:10.127004 25379 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:42221
I20260929 18:43:10.127029 25446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:42221 every 8 connection(s)
I20260929 18:43:10.128017 25379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:10.131014 25447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:10.136726 25447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832: Bootstrap starting.
I20260929 18:43:10.137050 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25379
I20260929 18:43:10.137189 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:43:10.137468 11739 external_mini_cluster.cc:1428] Setting key c4d89f29e54484999bb2ed06e8d0a5b9
I20260929 18:43:10.139690 25447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:10.140619 25447 log.cc:824] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:10.143070 25447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832: No bootstrap required, opened a new log
Sep 29 18:43:10 dist-test-slave-cgv7 krb5kdc[25366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:10.147298 25447 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } }
I20260929 18:43:10.147581 25447 raft_consensus.cc:374] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:10.147651 25447 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb765236f0e4212948bf442aac59832, State: Initialized, Role: FOLLOWER
I20260929 18:43:10.148183 25447 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } }
I20260929 18:43:10.148321 25447 raft_consensus.cc:388] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:10.148414 25447 raft_consensus.cc:482] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:10.148536 25447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:10.149890 25447 raft_consensus.cc:504] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } }
I20260929 18:43:10.150307 25447 leader_election.cc:302] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aeb765236f0e4212948bf442aac59832; no voters:
I20260929 18:43:10.150732 25447 leader_election.cc:288] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:43:10.150861 25452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:10.151113 25452 raft_consensus.cc:686] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [term 1 LEADER]: Becoming Leader. State: Replica: aeb765236f0e4212948bf442aac59832, State: Running, Role: LEADER
I20260929 18:43:10.151494 25452 consensus_queue.cc:237] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } }
I20260929 18:43:10.152258 25447 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:10.152709 25452 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeb765236f0e4212948bf442aac59832. Latest consensus state: current_term: 1 leader_uuid: "aeb765236f0e4212948bf442aac59832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } } }
I20260929 18:43:10.152904 25452 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:10.153177 25453 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aeb765236f0e4212948bf442aac59832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb765236f0e4212948bf442aac59832" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42221 } } }
I20260929 18:43:10.153380 25457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:10.154088 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:10.138953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.11.118.254:42221)
0929 18:43:10.139425 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:10.139445 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:10.139487 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:43:10.140384 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:10.140414 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:10.140859 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:10.141219 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:10.142079 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:10.143558 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:10.144448 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:10.145164 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:10.145340 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:10.148167 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:10.148172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:10.148185 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:10.148213 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:10.150309 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:10.150806 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:10.150814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:10.150820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:10.150975 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:10.151400 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:10.151404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:10.151406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:10.151932 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:10.152145 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:10.152490 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 18:43:10.152971 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":128,"threads_started":1}
I20260929 18:43:10.153318 25453 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:10.157311 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:43:10.157908 25457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:10.163331 25457 catalog_manager.cc:1347] Generated new cluster ID: 4aef122795c74b538b4e7d77b2402f9c
I20260929 18:43:10.163403 25457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:10.173604 25457 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:43:10.174682 25457 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:10.184774 25457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aeb765236f0e4212948bf442aac59832: Generated new TSK 0
I20260929 18:43:10.185454 25457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:43:10.213655 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25379
2026-09-29T18:43:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1461 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:43:10 dist-test-slave-cgv7 krb5kdc[25474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:43:10 dist-test-slave-cgv7 krb5kdc[25474](info): set up 2 sockets
Sep 29 18:43:10 dist-test-slave-cgv7 krb5kdc[25474](info): commencing operation
krb5kdc: starting...
W20260929 18:43:11.258785 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:43:11 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:43:11.386864 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:46165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:11.499287 25487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:11.499656 25487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:11.499743 25487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:11.503880 25487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:11.503997 25487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:11.504037 25487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:11.504076 25487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:11.504112 25487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:11.509596 25487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:46165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:11.510820 25487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:11.511895 25487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:43:11.520895 25487 server_base.cc:1034] running on GCE node
W20260929 18:43:11.520853 25493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:11.520902 25492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:11.520853 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:11.521767 25487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:11.523001 25487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:11.524174 25487 hybrid_clock.cc:648] HybridClock initialized: now 1790707391524153 us; error 34 us; skew 500 ppm
Sep 29 18:43:11 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707391, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:11.532757 25487 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:11.533937 25487 webserver.cc:460] Webserver started at http://127.11.118.254:45365/ using document root <none> and password file <none>
I20260929 18:43:11.534644 25487 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:11.534718 25487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:11.534953 25487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:11.537184 25487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "41acffe7152d4f6ab5751c3d2233f4f1"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "0a71582d8ba585913712519f68c75807"
server_key_iv: "eb599d973f45d901b8ffdbc42bb2c337"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.537807 25487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "41acffe7152d4f6ab5751c3d2233f4f1"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "0a71582d8ba585913712519f68c75807"
server_key_iv: "eb599d973f45d901b8ffdbc42bb2c337"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.541922 25487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260929 18:43:11.544555 25502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:11.545873 25487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:43:11.546022 25487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "41acffe7152d4f6ab5751c3d2233f4f1"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "0a71582d8ba585913712519f68c75807"
server_key_iv: "eb599d973f45d901b8ffdbc42bb2c337"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.546146 25487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:11.561288 25487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:11.564970 25487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:11.565222 25487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:11.572911 25487 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:46165
I20260929 18:43:11.572960 25554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:46165 every 8 connection(s)
I20260929 18:43:11.573947 25487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:11.576802 25555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:11.582525 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap starting.
I20260929 18:43:11.583904 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25487
I20260929 18:43:11.584038 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:43:11.584334 11739 external_mini_cluster.cc:1428] Setting key 205b7207a18fafbb1d387bb542ed722d
I20260929 18:43:11.585515 25555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:11.586308 25555 log.cc:824] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:11.588647 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: No bootstrap required, opened a new log
I20260929 18:43:11.593320 25555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:11.593585 25555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 29 18:43:11 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:11.593664 25555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Initialized, Role: FOLLOWER
I20260929 18:43:11.594162 25555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:11.594333 25555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:11.594427 25555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:11.594529 25555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:11.595484 25555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:11.595842 25555 leader_election.cc:302] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41acffe7152d4f6ab5751c3d2233f4f1; no voters:
I20260929 18:43:11.596215 25555 leader_election.cc:288] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:43:11.596290 25560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:11.596485 25560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 LEADER]: Becoming Leader. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Running, Role: LEADER
I20260929 18:43:11.596856 25560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:11.597585 25555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:11.598637 25562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41acffe7152d4f6ab5751c3d2233f4f1. Latest consensus state: current_term: 1 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:11.598779 25562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:11.599124 25561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:11.599265 25561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:11.600230 25569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:11.600610 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:11.585493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.11.118.254:46165)
0929 18:43:11.585979 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:11.585995 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:11.586030 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:43:11.586819 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:11.586844 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:11.587255 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:11.587545 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:11.588459 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.589504 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.590721 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.591208 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.591327 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:11.594355 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:11.594361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:11.594380 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:11.594439 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:11.596582 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:11.597147 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:11.597151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:11.597156 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:11.597269 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:11.597581 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:11.597585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:11.597587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:11.598190 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:11.599164 (+ 974us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:11.599314 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 18:43:11.599620 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":166,"threads_started":1}
I20260929 18:43:11.603396 25569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:11.609041 25569 catalog_manager.cc:1347] Generated new cluster ID: cd759edd564f4bd6ba7fa75ad02e839e
I20260929 18:43:11.609122 25569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:11.619002 25569 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:43:11.619973 25569 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:11.627651 25569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Generated new TSK 0
I20260929 18:43:11.628399 25569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:43:11.665875 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:11.794776 25581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:11.795064 25581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:11.795109 25581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:11.799391 25581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:11.799456 25581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:11.799566 25581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:11.805148 25581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:11.806285 25581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:11.807335 25581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:11.817127 25587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:11.817137 25586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:11.817353 25581 server_base.cc:1034] running on GCE node
W20260929 18:43:11.817438 25589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:11.817811 25581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:11.818540 25581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:11.819741 25581 hybrid_clock.cc:648] HybridClock initialized: now 1790707391819699 us; error 31 us; skew 500 ppm
Sep 29 18:43:11 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707391, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:11.822777 25581 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:11.823753 25581 webserver.cc:460] Webserver started at http://127.11.118.193:45069/ using document root <none> and password file <none>
I20260929 18:43:11.824347 25581 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:11.824455 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:11.824679 25581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:11.826889 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "4f971017b6af46fda779b6da2f72b85f"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "64383def0c02c267b4f17db06b1742f2"
server_key_iv: "79b6969d94c2f63f30ad7b3023a80fde"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.827490 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "4f971017b6af46fda779b6da2f72b85f"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "64383def0c02c267b4f17db06b1742f2"
server_key_iv: "79b6969d94c2f63f30ad7b3023a80fde"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.831398 25581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:11.834077 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:11.835386 25581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:43:11.835538 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "4f971017b6af46fda779b6da2f72b85f"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "64383def0c02c267b4f17db06b1742f2"
server_key_iv: "79b6969d94c2f63f30ad7b3023a80fde"
server_key_version: "encryptionkey@0"
I20260929 18:43:11.835657 25581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:11.848124 25581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:11.851415 25581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:11.851622 25581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:11.852241 25581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:11.853138 25581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:11.853214 25581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:11.853278 25581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:11.853327 25581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:11.862782 25581 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:44585
I20260929 18:43:11.862799 25709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:44585 every 8 connection(s)
I20260929 18:43:11.863793 25581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
Sep 29 18:43:11 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707391, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:11.873598 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25581
I20260929 18:43:11.873768 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:43:11.874073 11739 external_mini_cluster.cc:1428] Setting key 4e1217c52628e84d9edb579a413d68d8
I20260929 18:43:11.878069 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:11.865737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:59237 (local address 127.11.118.254:46165)
0929 18:43:11.865885 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:11.865889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:11.867049 (+ 1160us) server_negotiation.cc:407] Connection header received
0929 18:43:11.867972 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:11.867975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:11.868033 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:11.868127 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:11.869366 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.870029 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.871379 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.871648 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.871735 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:11.874394 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:11.874398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:11.874401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:11.874434 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:11.875945 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:11.876504 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:11.876508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:11.876510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:11.876592 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:11.876961 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:11.876964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:11.876966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:11.877149 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:11.877259 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:11.877777 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 18:43:11.877905 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:43:11.878960 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:11.866120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:59237)
0929 18:43:11.866881 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:11.866926 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:11.867748 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:11.868307 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:11.868315 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:11.868696 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:11.869192 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:11.869207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.870182 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.870186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:11.871234 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:11.871246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:11.871794 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:11.871796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:11.871965 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:11.872536 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:11.872553 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:11.874191 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:11.876086 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:11.876092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:11.876111 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:11.876393 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:11.876687 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:11.876691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:11.876693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:11.876853 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:11.877258 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:11.877266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:11.877526 (+ 260us) client_negotiation.cc:769] Sending connection context
0929 18:43:11.877716 (+ 190us) client_negotiation.cc:240] Negotiation successful
0929 18:43:11.878073 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":196,"threads_started":1}
I20260929 18:43:11.880251 25710 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:11.880527 25710 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:11.881034 25710 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
I20260929 18:43:11.882856 25519 ts_manager.cc:194] Registered new tserver with Master: 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585)
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:43:11.884411 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:59237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:43:11.901583 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:12.018429 25715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:12.018713 25715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:12.018795 25715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:12.022956 25715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:12.023026 25715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:12.023139 25715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:12.028752 25715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:12.029922 25715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:12.031212 25715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:12.041756 25723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:12.042142 25721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.042372 25715 server_base.cc:1034] running on GCE node
W20260929 18:43:12.042529 25720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.043043 25715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:12.043747 25715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:12.044977 25715 hybrid_clock.cc:648] HybridClock initialized: now 1790707392044927 us; error 71 us; skew 500 ppm
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:12.048230 25715 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:12.049302 25715 webserver.cc:460] Webserver started at http://127.11.118.194:37885/ using document root <none> and password file <none>
I20260929 18:43:12.049943 25715 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:12.050041 25715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:12.050356 25715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:12.052508 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "dfe99c390d32475997cf7c1b9432d0c6"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "11b56ea7956d90d07d19efe0acdb9835"
server_key_iv: "a579847d36587569d69ea14790086ef8"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.053097 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "dfe99c390d32475997cf7c1b9432d0c6"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "11b56ea7956d90d07d19efe0acdb9835"
server_key_iv: "a579847d36587569d69ea14790086ef8"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.056967 25715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 18:43:12.059508 25730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.060806 25715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:43:12.061004 25715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "dfe99c390d32475997cf7c1b9432d0c6"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "11b56ea7956d90d07d19efe0acdb9835"
server_key_iv: "a579847d36587569d69ea14790086ef8"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.061127 25715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:12.077702 25715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:12.080917 25715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:12.081135 25715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:12.081744 25715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:12.082698 25715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:12.082770 25715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.082844 25715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:12.082894 25715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.092257 25715 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41265
I20260929 18:43:12.092281 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41265 every 8 connection(s)
I20260929 18:43:12.093183 25715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:12.098476 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25715
I20260929 18:43:12.098629 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:43:12.098934 11739 external_mini_cluster.cc:1428] Setting key 3b9f448dbf47bafa5733c5ca86f1b21f
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:12.108481 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.095033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:49817 (local address 127.11.118.254:46165)
0929 18:43:12.095169 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.095173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.096032 (+ 859us) server_negotiation.cc:407] Connection header received
0929 18:43:12.097341 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.097344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.097404 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.097478 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:12.099300 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.099937 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.101284 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.101515 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.101631 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.104353 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:12.104360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:12.104365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:12.104397 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:12.106235 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.106766 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.106769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.106771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.106880 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.107225 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.107228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.107230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.107492 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:12.107610 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.108128 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.108268 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:43:12.109309 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.095433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:49817)
0929 18:43:12.095879 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.095921 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.097146 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.097716 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.097722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.098097 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:12.098934 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.098958 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.100072 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.100075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.101157 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.101167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.101677 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.101681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.101869 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.102539 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:12.102557 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:12.104153 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:12.106383 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.106389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.106401 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.106661 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.106983 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.106985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.106987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.107125 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.107601 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:12.107606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:12.107871 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.108035 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.108318 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260929 18:43:12.110895 25844 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:12.111187 25844 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:12.111728 25844 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:43:12.112994 25519 ts_manager.cc:194] Registered new tserver with Master: dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265)
I20260929 18:43:12.113838 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:49817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:43:12.130379 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:12.246449 25849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:12.246726 25849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:12.246773 25849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:12.251003 25849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:12.251060 25849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:12.251120 25849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:12.256537 25849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:12.257604 25849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:12.258621 25849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:12.269019 25855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.269034 25849 server_base.cc:1034] running on GCE node
W20260929 18:43:12.269054 25857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:12.269019 25854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.269723 25849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:12.270462 25849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:12.271646 25849 hybrid_clock.cc:648] HybridClock initialized: now 1790707392271628 us; error 39 us; skew 500 ppm
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:12.274434 25849 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:12.275379 25849 webserver.cc:460] Webserver started at http://127.11.118.195:44253/ using document root <none> and password file <none>
I20260929 18:43:12.275964 25849 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:12.276057 25849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:12.276319 25849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:12.278442 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "5b1ae367c05d4a398733d23182cff608"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "2c92ea2570edc412c49ea7ba7c536221"
server_key_iv: "c0612d5e30f35096364883135d2d9dbe"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.279039 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "5b1ae367c05d4a398733d23182cff608"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "2c92ea2570edc412c49ea7ba7c536221"
server_key_iv: "c0612d5e30f35096364883135d2d9dbe"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.282861 25849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:12.285359 25864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.286772 25849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:12.286929 25849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "5b1ae367c05d4a398733d23182cff608"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "2c92ea2570edc412c49ea7ba7c536221"
server_key_iv: "c0612d5e30f35096364883135d2d9dbe"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.287051 25849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:12.304088 25849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:12.307386 25849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:12.307605 25849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:12.308233 25849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:12.309077 25849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:12.309150 25849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.309221 25849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:12.309264 25849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.318789 25849 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:34243
I20260929 18:43:12.318820 25977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:34243 every 8 connection(s)
I20260929 18:43:12.319713 25849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:12.327246 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 25849
I20260929 18:43:12.327374 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:43:12.327669 11739 external_mini_cluster.cc:1428] Setting key 06b8c00f5ac7ee38eeb48d905679480b
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:12.333653 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.321534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:45587 (local address 127.11.118.254:46165)
0929 18:43:12.321681 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.321685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.322751 (+ 1066us) server_negotiation.cc:407] Connection header received
0929 18:43:12.323626 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.323629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.323676 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.323748 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:12.324998 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.325538 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.326852 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.327080 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.327167 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.330207 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:12.330211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:12.330213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:12.330240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:12.331587 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.332105 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.332109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.332111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.332194 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.332542 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.332545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.332547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.332740 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:12.332842 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.333367 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.333490 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:43:12.334547 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.321915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:45587)
0929 18:43:12.322599 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.322643 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.323440 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.323934 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.323941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.324303 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:12.324791 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.324805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.325677 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.325681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.326721 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.326731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.327266 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.327273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.327517 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.328205 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:12.328247 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:12.330043 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:12.331709 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.331715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.331732 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.331997 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.332291 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.332294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.332296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.332439 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.332859 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:12.332866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:12.333115 (+ 249us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.333316 (+ 201us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.333651 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":108,"threads_started":1}
I20260929 18:43:12.335749 25978 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:12.335969 25978 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:12.336450 25978 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
I20260929 18:43:12.337417 25519 ts_manager.cc:194] Registered new tserver with Master: 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:12.337944 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:45587
I20260929 18:43:12.341756 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:12.351444 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.343823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.11.118.254:46165)
0929 18:43:12.344000 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.344005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.344160 (+ 155us) server_negotiation.cc:407] Connection header received
0929 18:43:12.344315 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.344318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.344365 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.344449 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:12.345161 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.345682 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.346642 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.346868 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.346976 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.348357 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:12.348361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:12.348366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:12.348396 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:12.349617 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.350121 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.350125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.350127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.350213 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.350538 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.350542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.350544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.350790 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:12.350936 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.351149 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.351268 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 18:43:12.354467 25519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:43:12.357605 25519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:43:12.373327 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.366888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:44585 (local address 127.0.0.1:46078)
0929 18:43:12.367787 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.367825 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.367989 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.368752 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.368758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.368780 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:12.368917 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.368924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.370093 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.370098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.371679 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.371690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.372703 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.372710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.372851 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.372895 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.373034 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.373169 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":75,"thread_start_us":99,"threads_started":1}
I20260929 18:43:12.373351 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.366981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.11.118.193:44585)
0929 18:43:12.367381 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.367399 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.367928 (+ 529us) server_negotiation.cc:407] Connection header received
0929 18:43:12.368154 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.368160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.368391 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.368550 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:12.369058 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.369956 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.371819 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.372556 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.372664 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.372766 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.373095 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.373198 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":107,"threads_started":1}
I20260929 18:43:12.374056 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.366951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:41265 (local address 127.0.0.1:60144)
0929 18:43:12.368232 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.368244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.368314 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.368909 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.368913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.368931 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:12.369031 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.369039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.370408 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.370411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.371680 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.371690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.373727 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.373730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.373832 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.373855 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.373896 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.373944 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"mutex_wait_us":41,"thread_start_us":84,"threads_started":1}
I20260929 18:43:12.374223 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.367439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34243 (local address 127.0.0.1:48488)
0929 18:43:12.368504 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.368516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.368590 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.369245 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.369248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.369264 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:12.369348 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.369353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.370933 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.370936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.372060 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.372071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.373609 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.373612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.373719 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.373753 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.373842 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.373920 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":8,"thread_start_us":99,"threads_started":1}
I20260929 18:43:12.374397 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.367742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.11.118.195:34243)
0929 18:43:12.368155 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.368167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.368695 (+ 528us) server_negotiation.cc:407] Connection header received
0929 18:43:12.368774 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.368780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.368953 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.369090 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:12.369477 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.370563 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.372212 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.372942 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.373068 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.373181 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.373882 (+ 701us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.373985 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":116,"threads_started":1}
I20260929 18:43:12.375078 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.367723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.11.118.194:41265)
0929 18:43:12.368444 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.368457 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.368485 (+ 28us) server_negotiation.cc:407] Connection header received
0929 18:43:12.368567 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.368572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.368751 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.368919 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:12.369165 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.370251 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.372822 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.373601 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.374539 (+ 938us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.374657 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.374705 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.374789 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":134,"threads_started":1}
I20260929 18:43:12.376881 25644 tablet_service.cc:1467] Processing CreateTablet for tablet 6f5df495421d4884b1a603a770bb6d4f (DEFAULT_TABLE table=test-table [id=4c34b968aba64ca8a82410840c2210ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:12.377233 25778 tablet_service.cc:1467] Processing CreateTablet for tablet 6f5df495421d4884b1a603a770bb6d4f (DEFAULT_TABLE table=test-table [id=4c34b968aba64ca8a82410840c2210ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:12.377300 25912 tablet_service.cc:1467] Processing CreateTablet for tablet 6f5df495421d4884b1a603a770bb6d4f (DEFAULT_TABLE table=test-table [id=4c34b968aba64ca8a82410840c2210ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:12.378027 25778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f5df495421d4884b1a603a770bb6d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:12.378049 25644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f5df495421d4884b1a603a770bb6d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:12.378189 25912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f5df495421d4884b1a603a770bb6d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:12.384495 25995 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Bootstrap starting.
I20260929 18:43:12.385627 25997 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Bootstrap starting.
I20260929 18:43:12.385949 25996 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Bootstrap starting.
I20260929 18:43:12.387326 25995 tablet_bootstrap.cc:654] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:12.387935 25996 tablet_bootstrap.cc:654] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:12.387935 25997 tablet_bootstrap.cc:654] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:12.388162 25995 log.cc:824] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:12.388845 25996 log.cc:824] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:12.389104 25997 log.cc:824] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:12.390597 25995 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: No bootstrap required, opened a new log
I20260929 18:43:12.390887 25995 ts_tablet_manager.cc:1397] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 18:43:12.390975 25996 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: No bootstrap required, opened a new log
I20260929 18:43:12.391163 25996 ts_tablet_manager.cc:1397] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:43:12.391184 25997 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: No bootstrap required, opened a new log
I20260929 18:43:12.391347 25997 ts_tablet_manager.cc:1397] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:43:12.394125 25995 raft_consensus.cc:348] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.394485 25995 raft_consensus.cc:374] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:12.394544 25995 raft_consensus.cc:729] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe99c390d32475997cf7c1b9432d0c6, State: Initialized, Role: FOLLOWER
I20260929 18:43:12.395022 25995 consensus_queue.cc:260] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.395293 25997 raft_consensus.cc:348] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.395579 25997 raft_consensus.cc:374] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:12.395648 25997 raft_consensus.cc:729] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1ae367c05d4a398733d23182cff608, State: Initialized, Role: FOLLOWER
I20260929 18:43:12.395680 25996 raft_consensus.cc:348] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.395893 25844 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
I20260929 18:43:12.395917 25996 raft_consensus.cc:374] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:12.395973 25996 raft_consensus.cc:729] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f971017b6af46fda779b6da2f72b85f, State: Initialized, Role: FOLLOWER
I20260929 18:43:12.396147 25995 ts_tablet_manager.cc:1428] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:43:12.396437 25996 consensus_queue.cc:260] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.397154 25710 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
I20260929 18:43:12.397255 25997 consensus_queue.cc:260] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.397480 25996 ts_tablet_manager.cc:1428] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:43:12.398016 25978 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
I20260929 18:43:12.398227 25997 ts_tablet_manager.cc:1428] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 18:43:12.436702 26001 raft_consensus.cc:482] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:12.436915 26001 raft_consensus.cc:504] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.438174 26001 leader_election.cc:288] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:12.442463 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.438545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:44585 (local address 127.11.118.194:46837)
0929 18:43:12.438690 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.438706 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.438822 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.439122 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.439124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.439142 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:12.439303 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.439311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.440222 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.440226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.441154 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.441163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.441940 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.441943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.442051 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.442077 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.442163 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.442262 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 18:43:12.442514 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.438587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:46837 (local address 127.11.118.193:44585)
0929 18:43:12.438780 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.438787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.438808 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:43:12.438953 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.438957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.439012 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.439104 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:12.439425 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.440110 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.441293 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.441831 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.441929 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.441970 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.442222 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.442384 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260929 18:43:12.442838 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.438728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:40615 (local address 127.11.118.195:34243)
0929 18:43:12.438862 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.438868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.439221 (+ 353us) server_negotiation.cc:407] Connection header received
0929 18:43:12.439359 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.439365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.439419 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.439486 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:12.439872 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.440489 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.441628 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.442221 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.442372 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.442418 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.442636 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.442726 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 18:43:12.443241 25664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f5df495421d4884b1a603a770bb6d4f" candidate_uuid: "dfe99c390d32475997cf7c1b9432d0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f971017b6af46fda779b6da2f72b85f" is_pre_election: true
I20260929 18:43:12.443236 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.438546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34243 (local address 127.11.118.194:40615)
0929 18:43:12.439099 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.439113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.439232 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.439515 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.439518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.439535 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:12.439671 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.439679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.440597 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.440600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.441496 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.441506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.442364 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.442368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.442468 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.442490 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.443031 (+ 541us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.443106 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":138,"thread_start_us":145,"threads_started":1}
I20260929 18:43:12.443589 25664 raft_consensus.cc:2454] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe99c390d32475997cf7c1b9432d0c6 in term 0.
I20260929 18:43:12.444191 25932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f5df495421d4884b1a603a770bb6d4f" candidate_uuid: "dfe99c390d32475997cf7c1b9432d0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1ae367c05d4a398733d23182cff608" is_pre_election: true
I20260929 18:43:12.444516 25932 raft_consensus.cc:2454] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe99c390d32475997cf7c1b9432d0c6 in term 0.
I20260929 18:43:12.444998 25733 leader_election.cc:302] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f, 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6; no voters:
I20260929 18:43:12.445320 26001 raft_consensus.cc:2781] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:12.445428 26001 raft_consensus.cc:482] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:12.445492 26001 raft_consensus.cc:3037] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:12.446662 26001 raft_consensus.cc:504] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.447032 26001 leader_election.cc:288] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [CANDIDATE]: Term 1 election: Requested vote from peers 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:12.447484 25664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f5df495421d4884b1a603a770bb6d4f" candidate_uuid: "dfe99c390d32475997cf7c1b9432d0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f971017b6af46fda779b6da2f72b85f"
I20260929 18:43:12.447484 25932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f5df495421d4884b1a603a770bb6d4f" candidate_uuid: "dfe99c390d32475997cf7c1b9432d0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1ae367c05d4a398733d23182cff608"
I20260929 18:43:12.447616 25932 raft_consensus.cc:3037] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:12.447616 25664 raft_consensus.cc:3037] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:12.448932 25932 raft_consensus.cc:2454] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe99c390d32475997cf7c1b9432d0c6 in term 1.
I20260929 18:43:12.448932 25664 raft_consensus.cc:2454] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe99c390d32475997cf7c1b9432d0c6 in term 1.
I20260929 18:43:12.449239 25734 leader_election.cc:302] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6; no voters:
I20260929 18:43:12.449446 26001 raft_consensus.cc:2781] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:12.449652 26001 raft_consensus.cc:686] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 1 LEADER]: Becoming Leader. State: Replica: dfe99c390d32475997cf7c1b9432d0c6, State: Running, Role: LEADER
I20260929 18:43:12.450033 26001 consensus_queue.cc:237] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.453187 25519 catalog_manager.cc:5495] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "dfe99c390d32475997cf7c1b9432d0c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } health_report { overall_health: UNKNOWN } } }
I20260929 18:43:12.471814 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.468443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.11.118.194:41265)
0929 18:43:12.468564 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.468568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.468582 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:12.468653 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.468655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.468712 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.468814 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:12.469048 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.469576 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.470442 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.470663 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.470744 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.471062 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:12.471482 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:12.471576 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.471634 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.471683 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 18:43:12.477780 25664 raft_consensus.cc:1261] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Refusing update from remote peer dfe99c390d32475997cf7c1b9432d0c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:12.477773 25932 raft_consensus.cc:1261] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Refusing update from remote peer dfe99c390d32475997cf7c1b9432d0c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:12.478538 26005 consensus_queue.cc:1035] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:12.478741 26001 consensus_queue.cc:1035] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:12.487201 26009 mvcc.cc:205] Tried to move back new op lower bound from 7334737479581229056 to 7334737479478312960. Current Snapshot: MvccSnapshot[applied={T|T < 7334737479581229056}]
I20260929 18:43:12.488970 26010 mvcc.cc:205] Tried to move back new op lower bound from 7334737479581229056 to 7334737479478312960. Current Snapshot: MvccSnapshot[applied={T|T < 7334737479581229056}]
I20260929 18:43:12.493703 25518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:12.493847 25518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:12.496193 25518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Sending DeleteTablet for 3 replicas of tablet 6f5df495421d4884b1a603a770bb6d4f
I20260929 18:43:12.496892 25644 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f5df495421d4884b1a603a770bb6d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:12 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46078
I20260929 18:43:12.496927 25778 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f5df495421d4884b1a603a770bb6d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:12 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:60144
I20260929 18:43:12.497409 26017 tablet_replica.cc:330] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: stopping tablet replica
I20260929 18:43:12.497597 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25581
I20260929 18:43:12.497586 25912 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f5df495421d4884b1a603a770bb6d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:12 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48488
I20260929 18:43:12.498466 26017 raft_consensus.cc:2229] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:43:12.498929 26017 raft_consensus.cc:2258] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:12.499763 26019 tablet_replica.cc:330] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: stopping tablet replica
I20260929 18:43:12.500089 26019 raft_consensus.cc:2229] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:12.500402 26019 raft_consensus.cc:2258] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:12.500705 26017 ts_tablet_manager.cc:1905] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:12.501854 26019 ts_tablet_manager.cc:1905] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:12.503579 26017 ts_tablet_manager.cc:1918] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:12.503743 26017 log.cc:1196] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/6f5df495421d4884b1a603a770bb6d4f
I20260929 18:43:12.504145 26017 ts_tablet_manager.cc:1939] T 6f5df495421d4884b1a603a770bb6d4f P dfe99c390d32475997cf7c1b9432d0c6: Deleting consensus metadata
I20260929 18:43:12.504647 26019 ts_tablet_manager.cc:1918] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:12.504743 26019 log.cc:1196] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6f5df495421d4884b1a603a770bb6d4f
I20260929 18:43:12.505067 26019 ts_tablet_manager.cc:1939] T 6f5df495421d4884b1a603a770bb6d4f P 5b1ae367c05d4a398733d23182cff608: Deleting consensus metadata
I20260929 18:43:12.505826 25504 catalog_manager.cc:4841] TS dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265): tablet 6f5df495421d4884b1a603a770bb6d4f (table test-table [id=4c34b968aba64ca8a82410840c2210ef]) successfully deleted
W20260929 18:43:12.505894 25505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:44585 (error 108)
I20260929 18:43:12.506199 25506 catalog_manager.cc:4841] TS 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): tablet 6f5df495421d4884b1a603a770bb6d4f (table test-table [id=4c34b968aba64ca8a82410840c2210ef]) successfully deleted
I20260929 18:43:12.506723 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25715
I20260929 18:43:12.507095 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.506752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:44585 (local address 127.0.0.1:46082)
0929 18:43:12.506928 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.507001 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260929 18:43:12.507650 25505 catalog_manager.cc:4584] TS 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f5df495421d4884b1a603a770bb6d4f: Network error: Client connection negotiation failed: client connection to 127.11.118.193:44585: connect: Connection refused (error 111)
I20260929 18:43:12.512874 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25849
I20260929 18:43:12.518329 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 25487
I20260929 18:43:12.524827 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--webserver_interface=127.11.118.254
--webserver_port=45365
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:46165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:12.648865 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:12.649152 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:12.649194 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:12.653390 26021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:12.653460 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:12.653478 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:12.653496 26021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:12.653512 26021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:12.659019 26021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:46165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=45365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:12.660058 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:12.661026 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:12.674469 26030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:12.674494 26027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.674652 26021 server_base.cc:1034] running on GCE node
W20260929 18:43:12.674472 26028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.675503 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:12.676787 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:12.677999 26021 hybrid_clock.cc:648] HybridClock initialized: now 1790707392677965 us; error 47 us; skew 500 ppm
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:12.680965 26021 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:12.681878 26021 webserver.cc:460] Webserver started at http://127.11.118.254:45365/ using document root <none> and password file <none>
I20260929 18:43:12.682523 26021 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:12.682598 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:12.686667 26021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:43:12.688954 26037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.690415 26021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:12.690565 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "41acffe7152d4f6ab5751c3d2233f4f1"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "0a71582d8ba585913712519f68c75807"
server_key_iv: "eb599d973f45d901b8ffdbc42bb2c337"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.691241 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:12.704209 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:12.707494 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:12.707713 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:12.714826 26021 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:46165
I20260929 18:43:12.715099 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:46165 every 8 connection(s)
I20260929 18:43:12.715996 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:12.720388 26090 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:43:12.721741 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26021
I20260929 18:43:12.722048 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap starting.
I20260929 18:43:12.722226 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:44585
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=45069
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:12.739907 26090 log.cc:824] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:12.748819 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:12.749202 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap complete.
I20260929 18:43:12.753227 26090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:12.753613 26090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Initialized, Role: FOLLOWER
I20260929 18:43:12.754141 26090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:12.754254 26090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:12.754369 26090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:12.754484 26090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:43:12.756364 26090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:12.756702 26090 leader_election.cc:302] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41acffe7152d4f6ab5751c3d2233f4f1; no voters:
I20260929 18:43:12.757038 26090 leader_election.cc:288] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 18:43:12.757118 26094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 18:43:12.757321 26094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 LEADER]: Becoming Leader. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Running, Role: LEADER
I20260929 18:43:12.757606 26094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:12.758253 26090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:12.758911 26095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:12.759035 26095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:12.759459 26096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41acffe7152d4f6ab5751c3d2233f4f1. Latest consensus state: current_term: 2 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:12.759604 26096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:12.759934 26101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:12.764654 26101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:12.765582 26101 catalog_manager.cc:1259] Loaded cluster ID: cd759edd564f4bd6ba7fa75ad02e839e
I20260929 18:43:12.765643 26101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:12.767231 26101 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 18:43:12.771814 26101 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 18:43:12.777290 26101 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:12.778157 26101 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Loaded TSK: 0
I20260929 18:43:12.779419 26101 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:43:12.842037 26092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:12.842448 26092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:12.842536 26092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:12.847126 26092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:12.847203 26092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:12.847267 26092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:12.852841 26092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=45069
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:12.853873 26092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:12.855005 26092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:12.865962 26117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:12.865984 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:12.866353 26118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:12.866932 26092 server_base.cc:1034] running on GCE node
I20260929 18:43:12.867412 26092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:12.868216 26092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:12.869433 26092 hybrid_clock.cc:648] HybridClock initialized: now 1790707392869394 us; error 46 us; skew 500 ppm
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:12.873081 26092 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:12.874244 26092 webserver.cc:460] Webserver started at http://127.11.118.193:45069/ using document root <none> and password file <none>
I20260929 18:43:12.875005 26092 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:12.875123 26092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:12.880312 26092 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 18:43:12.882752 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:12.884156 26092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:12.884310 26092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "4f971017b6af46fda779b6da2f72b85f"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "64383def0c02c267b4f17db06b1742f2"
server_key_iv: "79b6969d94c2f63f30ad7b3023a80fde"
server_key_version: "encryptionkey@0"
I20260929 18:43:12.884867 26092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:12.900867 26092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:12.904125 26092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:12.904349 26092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:12.904989 26092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:12.906122 26134 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 18:43:12.911413 26092 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 18:43:12.911500 26092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260929 18:43:12.911593 26092 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 18:43:12.913259 26092 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 18:43:12.913335 26092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 18:43:12.913604 26134 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Bootstrap starting.
I20260929 18:43:12.923566 26092 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:44585
I20260929 18:43:12.923668 26241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:44585 every 8 connection(s)
I20260929 18:43:12.924552 26092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:12.929412 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26092
I20260929 18:43:12.929953 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:41265
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=37885
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:12.934639 26134 log.cc:824] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Log is configured to *not* fsync() on all Append() calls
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:12.944214 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.926756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36415 (local address 127.11.118.254:46165)
0929 18:43:12.927165 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.927185 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.927559 (+ 374us) server_negotiation.cc:407] Connection header received
0929 18:43:12.928955 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.928966 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.929425 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.929798 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:12.931605 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.933326 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.935368 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.935935 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.936134 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.938908 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:12.938913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:12.938925 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:12.938953 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:12.940629 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.941184 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.941189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.941193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.941308 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.941666 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.941676 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.941678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.942066 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:12.942216 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.943091 (+ 875us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.943467 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":139,"threads_started":1}
I20260929 18:43:12.945084 26134 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:12.945530 26134 tablet_bootstrap.cc:492] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Bootstrap complete.
I20260929 18:43:12.945708 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.926748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:36415)
0929 18:43:12.927348 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.927426 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.928554 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.929913 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.929921 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.930674 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:12.931394 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.931416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.933525 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.933532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.935204 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.935219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.936093 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.936099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.936290 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.936911 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:12.936927 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:12.938665 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:12.940768 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.940774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.940786 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.941048 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.941418 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.941421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.941423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.941562 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.942187 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:12.942194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:12.942556 (+ 362us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.944513 (+ 1957us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.944855 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":142,"threads_started":1}
I20260929 18:43:12.947024 26134 ts_tablet_manager.cc:1397] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Time spent bootstrapping tablet: real 0.034s user 0.015s sys 0.017s
I20260929 18:43:12.948026 26242 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:12.948282 26242 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:12.948781 26242 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
I20260929 18:43:12.950184 26134 raft_consensus.cc:348] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.950748 26134 raft_consensus.cc:729] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f971017b6af46fda779b6da2f72b85f, State: Initialized, Role: FOLLOWER
I20260929 18:43:12.951223 26134 consensus_queue.cc:260] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } } peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } }
I20260929 18:43:12.951290 26054 ts_manager.cc:194] Registered new tserver with Master: 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585)
I20260929 18:43:12.952234 26134 ts_tablet_manager.cc:1428] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260929 18:43:12.955159 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:36415
I20260929 18:43:12.957713 26242 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
Sep 29 18:43:12 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707392, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:43:12.966666 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.954224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:44585 (local address 127.0.0.1:46098)
0929 18:43:12.954694 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:12.954742 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:12.954874 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:12.956522 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:12.956528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:12.956970 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:12.957100 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.957108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.958405 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.958409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.959358 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:12.959371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.960132 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.960135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:12.960225 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.960824 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:12.960837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:12.962516 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:12.964635 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.964644 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.964649 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.964926 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.965245 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:12.965247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:12.965248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:12.965346 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:12.965934 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:12.965944 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:12.966120 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 18:43:12.966254 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 18:43:12.966519 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":157,"threads_started":1}
I20260929 18:43:12.966765 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:12.954367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.11.118.193:44585)
0929 18:43:12.955979 (+ 1612us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:12.955991 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:12.956013 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:43:12.956072 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:12.956078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:12.956343 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:12.956485 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:12.957242 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.958225 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.959503 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:12.960012 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:12.960122 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:12.962681 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:12.962693 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:12.962712 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:12.962739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:12.964505 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.965045 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.965051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.965053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.965150 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:12.965435 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:12.965438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:12.965440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:12.965760 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:12.965894 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:12.966406 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 18:43:12.966591 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1512,"thread_start_us":98,"threads_started":1}
I20260929 18:43:12.967900 26176 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f5df495421d4884b1a603a770bb6d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:12 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46098
I20260929 18:43:12.968786 26251 tablet_replica.cc:330] stopping tablet replica
I20260929 18:43:12.969108 26251 raft_consensus.cc:2229] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:12.969374 26251 raft_consensus.cc:2258] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:12.970432 26251 ts_tablet_manager.cc:1905] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:12.974102 26251 ts_tablet_manager.cc:1918] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:12.974193 26251 log.cc:1196] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/6f5df495421d4884b1a603a770bb6d4f
I20260929 18:43:12.974510 26251 ts_tablet_manager.cc:1939] T 6f5df495421d4884b1a603a770bb6d4f P 4f971017b6af46fda779b6da2f72b85f: Deleting consensus metadata
I20260929 18:43:12.976290 26040 catalog_manager.cc:4841] TS 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585): tablet 6f5df495421d4884b1a603a770bb6d4f (table test-table [id=4c34b968aba64ca8a82410840c2210ef]) successfully deleted
W20260929 18:43:13.053471 26246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:13.053766 26246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:13.053841 26246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:13.057874 26246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:13.057961 26246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:13.058058 26246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:13.063393 26246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:41265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=37885
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:13.064503 26246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:13.065524 26246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:13.075515 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:13.075528 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.075515 26246 server_base.cc:1034] running on GCE node
W20260929 18:43:13.075734 26259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.076174 26246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:13.076774 26246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:13.077950 26246 hybrid_clock.cc:648] HybridClock initialized: now 1790707393077918 us; error 46 us; skew 500 ppm
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:13.080960 26246 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:13.081861 26246 webserver.cc:460] Webserver started at http://127.11.118.194:37885/ using document root <none> and password file <none>
I20260929 18:43:13.082523 26246 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:13.082629 26246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:13.087036 26246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:43:13.089550 26266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.090986 26246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:13.091133 26246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "dfe99c390d32475997cf7c1b9432d0c6"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "11b56ea7956d90d07d19efe0acdb9835"
server_key_iv: "a579847d36587569d69ea14790086ef8"
server_key_version: "encryptionkey@0"
I20260929 18:43:13.091688 26246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:13.108439 26246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:13.111541 26246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:13.111747 26246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:13.112571 26246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:13.113560 26246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:13.113608 26246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.113682 26246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:13.113714 26246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.122726 26246 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41265
I20260929 18:43:13.122793 26379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41265 every 8 connection(s)
I20260929 18:43:13.123862 26246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:13.126740 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26246
I20260929 18:43:13.127226 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:34243
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=44253
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:13.138803 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.125800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:54997 (local address 127.11.118.254:46165)
0929 18:43:13.125985 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.125990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.126913 (+ 923us) server_negotiation.cc:407] Connection header received
0929 18:43:13.128381 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.128384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.128440 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.128528 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:13.129726 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.130322 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.131505 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.131762 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.131881 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.134764 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:13.134767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:13.134769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:13.134877 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:13.136452 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.137085 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.137090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.137092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.137179 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.137532 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.137537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.137539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.137741 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:13.137854 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.138438 (+ 584us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.138611 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 18:43:13.139606 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.126223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:54997)
0929 18:43:13.126685 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.126728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.128139 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.128718 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.128725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.129086 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:13.129574 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.129587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.130468 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.130470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.131374 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.131382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.131921 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.131924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.132097 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.132787 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:13.132803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:13.134581 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:13.136605 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:13.136612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:13.136628 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:13.136974 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:13.137285 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:13.137288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:13.137290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:13.137434 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:13.137852 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:13.137859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:13.138117 (+ 258us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.138395 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.138757 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":132,"threads_started":1}
I20260929 18:43:13.140976 26380 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:13.141232 26380 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:13.141765 26380 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
I20260929 18:43:13.142990 26054 ts_manager.cc:194] Registered new tserver with Master: dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265)
I20260929 18:43:13.143869 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:54997
W20260929 18:43:13.242479 26383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:13.242758 26383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:13.242801 26383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:13.246848 26383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:13.246910 26383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:13.246968 26383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:13.252166 26383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:34243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=44253
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:13.253157 26383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:13.254096 26383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:13.263796 26391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.263859 26383 server_base.cc:1034] running on GCE node
W20260929 18:43:13.263808 26388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:13.263813 26389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.264451 26383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:13.265054 26383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:13.266213 26383 hybrid_clock.cc:648] HybridClock initialized: now 1790707393266197 us; error 25 us; skew 500 ppm
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:13.269021 26383 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:13.269870 26383 webserver.cc:460] Webserver started at http://127.11.118.195:44253/ using document root <none> and password file <none>
I20260929 18:43:13.270479 26383 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:13.270589 26383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:13.274418 26383 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:43:13.276657 26398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.278021 26383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:13.278164 26383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "5b1ae367c05d4a398733d23182cff608"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "2c92ea2570edc412c49ea7ba7c536221"
server_key_iv: "c0612d5e30f35096364883135d2d9dbe"
server_key_version: "encryptionkey@0"
I20260929 18:43:13.278779 26383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:13.295347 26383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:13.298584 26383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:13.298801 26383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:13.299405 26383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:13.300225 26383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:13.300271 26383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.300333 26383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:13.300366 26383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.309285 26383 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:34243
I20260929 18:43:13.309319 26511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:34243 every 8 connection(s)
I20260929 18:43:13.310349 26383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:13.313992 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26383
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:13.323066 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.314718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34636 (local address 127.11.118.254:46165)
0929 18:43:13.315139 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.315143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.315157 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:13.315237 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.315239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.315280 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.315378 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:13.316104 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.316712 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.317640 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.318009 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.318126 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.319597 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:13.319601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:13.319604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:13.319628 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:13.321166 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.321613 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.321617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.321619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.321711 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.322000 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.322004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.322005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.322220 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:13.322497 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.322693 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.322883 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":117,"threads_started":1}
I20260929 18:43:13.325194 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.312311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:57461 (local address 127.11.118.254:46165)
0929 18:43:13.312476 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.312480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.313278 (+ 798us) server_negotiation.cc:407] Connection header received
0929 18:43:13.314821 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.314827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.314887 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.314987 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:13.316336 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.316873 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.318183 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.318559 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.318671 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.321396 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:13.321404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:13.321406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:13.321426 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:13.322883 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.323553 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.323557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.323559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.323639 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:13.324069 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:13.324072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:13.324074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:13.324263 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:13.324340 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.324971 (+ 631us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.325088 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 18:43:13.326100 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.312655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:57461)
0929 18:43:13.313124 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.313166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.314590 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.315195 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.315207 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.315655 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:13.316188 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.316207 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.316995 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.316998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.318071 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.318083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.318721 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.318726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.318907 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.319466 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:13.319483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:13.321204 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:13.323042 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:13.323050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:13.323075 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:13.323448 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:13.323736 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:13.323739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:13.323740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:13.323968 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:13.324373 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:13.324379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:13.324689 (+ 310us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.324911 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.325213 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260929 18:43:13.327225 26512 heartbeater.cc:347] Connected to a master server at 127.11.118.254:46165
I20260929 18:43:13.327490 26512 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:13.327996 26512 heartbeater.cc:511] Master 127.11.118.254:46165 requested a full tablet report, sending...
I20260929 18:43:13.329027 26054 ts_manager.cc:194] Registered new tserver with Master: 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:13.329578 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:57461
I20260929 18:43:13.336635 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:13.340660 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.337168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.11.118.254:46165)
0929 18:43:13.337329 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.337334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.337492 (+ 158us) server_negotiation.cc:407] Connection header received
0929 18:43:13.337618 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.337620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.337713 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.337798 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:13.338002 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.338570 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.339380 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.339581 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.339655 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.339892 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:13.340288 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:13.340385 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.340447 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.340491 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:43:13.341357 26054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:43:13.344904 26054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:43:13.353929 26176 tablet_service.cc:1467] Processing CreateTablet for tablet e2b0f3ad4866487c8c0bc113129268ad (DEFAULT_TABLE table=test-table [id=a40ec4f809c34be3989f4522331c9f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:13.354617 26176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b0f3ad4866487c8c0bc113129268ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:13.357633 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.352142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:41265 (local address 127.0.0.1:60162)
0929 18:43:13.352452 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.352469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.352552 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.353632 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.353638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.353670 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:13.353851 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.353856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.355243 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.355247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.356153 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.356163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.357129 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.357133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.357245 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.357313 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.357397 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.357490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":53}
I20260929 18:43:13.357726 26134 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Bootstrap starting.
I20260929 18:43:13.357880 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.352742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.11.118.194:41265)
0929 18:43:13.353085 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.353097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.353122 (+ 25us) server_negotiation.cc:407] Connection header received
0929 18:43:13.353205 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.353212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.353466 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.353639 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:13.353981 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.355097 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.356285 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.356998 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.357131 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.357290 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.357509 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.357610 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":143,"threads_started":1}
I20260929 18:43:13.358639 26134 tablet_bootstrap.cc:654] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:13.358696 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.352376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34243 (local address 127.0.0.1:48494)
0929 18:43:13.352637 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.352649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.352746 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.353702 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.353704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.353716 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:13.353848 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.353855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.355459 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.355463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.356561 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.356572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.358242 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.358246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.358359 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.358382 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.358468 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.358541 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":11,"thread_start_us":107,"threads_started":1}
I20260929 18:43:13.358790 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.352848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.11.118.195:34243)
0929 18:43:13.353192 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.353202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.353242 (+ 40us) server_negotiation.cc:407] Connection header received
0929 18:43:13.353336 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.353341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.353531 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.354004 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:13.354203 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.355320 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.356704 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.358116 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.358256 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.358402 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.358495 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.358638 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":151,"threads_started":1}
I20260929 18:43:13.360877 26134 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: No bootstrap required, opened a new log
I20260929 18:43:13.360978 26134 ts_tablet_manager.cc:1397] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:13.361440 26134 raft_consensus.cc:348] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.361548 26134 raft_consensus.cc:374] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:13.361581 26134 raft_consensus.cc:729] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f971017b6af46fda779b6da2f72b85f, State: Initialized, Role: FOLLOWER
I20260929 18:43:13.361248 26314 tablet_service.cc:1467] Processing CreateTablet for tablet e2b0f3ad4866487c8c0bc113129268ad (DEFAULT_TABLE table=test-table [id=a40ec4f809c34be3989f4522331c9f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:13.361716 26134 consensus_queue.cc:260] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.361382 26446 tablet_service.cc:1467] Processing CreateTablet for tablet e2b0f3ad4866487c8c0bc113129268ad (DEFAULT_TABLE table=test-table [id=a40ec4f809c34be3989f4522331c9f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:13.361974 26134 ts_tablet_manager.cc:1428] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 18:43:13.362330 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b0f3ad4866487c8c0bc113129268ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:13.362506 26446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b0f3ad4866487c8c0bc113129268ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:13.370184 26525 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Bootstrap starting.
I20260929 18:43:13.370762 26526 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Bootstrap starting.
I20260929 18:43:13.373099 26525 tablet_bootstrap.cc:654] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:13.373205 26526 tablet_bootstrap.cc:654] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:13.373939 26526 log.cc:824] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:13.373939 26525 log.cc:824] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:13.376247 26526 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: No bootstrap required, opened a new log
I20260929 18:43:13.376236 26525 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: No bootstrap required, opened a new log
I20260929 18:43:13.376461 26526 ts_tablet_manager.cc:1397] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 18:43:13.376461 26525 ts_tablet_manager.cc:1397] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260929 18:43:13.380632 26526 raft_consensus.cc:348] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.380632 26525 raft_consensus.cc:348] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.380955 26526 raft_consensus.cc:374] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:13.380955 26525 raft_consensus.cc:374] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:13.381009 26525 raft_consensus.cc:729] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1ae367c05d4a398733d23182cff608, State: Initialized, Role: FOLLOWER
I20260929 18:43:13.381009 26526 raft_consensus.cc:729] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe99c390d32475997cf7c1b9432d0c6, State: Initialized, Role: FOLLOWER
I20260929 18:43:13.381521 26526 consensus_queue.cc:260] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.381521 26525 consensus_queue.cc:260] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.382355 26380 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
I20260929 18:43:13.382562 26512 heartbeater.cc:503] Master 127.11.118.254:46165 was elected leader, sending a full tablet report...
I20260929 18:43:13.382704 26525 ts_tablet_manager.cc:1428] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260929 18:43:13.382704 26526 ts_tablet_manager.cc:1428] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
W20260929 18:43:13.425997 26243 tablet.cc:2369] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:43:13.476912 26248 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:13.477134 26248 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.478284 26248 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:13.483736 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.478656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:40465 (local address 127.11.118.194:41265)
0929 18:43:13.478787 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.478792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.479053 (+ 261us) server_negotiation.cc:407] Connection header received
0929 18:43:13.479157 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.479160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.479216 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.479312 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:13.479772 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.480561 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.482378 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.483032 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.483179 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.483243 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.483526 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.483602 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:43:13.483773 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.478669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:38835 (local address 127.11.118.195:34243)
0929 18:43:13.478800 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.478805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.478971 (+ 166us) server_negotiation.cc:407] Connection header received
0929 18:43:13.479157 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.479160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.479212 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.479300 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:13.479773 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.480561 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.482388 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.483097 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.483204 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.483243 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.483526 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.483602 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:43:13.483808 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.478533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:41265 (local address 127.11.118.193:40465)
0929 18:43:13.478976 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.478989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.479061 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.479331 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.479334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.479375 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:13.479655 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.479662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.480680 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.480684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.482191 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.482204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.483176 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.483180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.483286 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.483404 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.483490 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.483618 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":296,"spinlock_wait_cycles":28672,"thread_start_us":75,"threads_started":1}
I20260929 18:43:13.483826 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.478529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34243 (local address 127.11.118.193:38835)
0929 18:43:13.478877 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.478894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.479022 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:13.479331 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:13.479334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:13.479376 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:13.479659 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.479664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.480680 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.480684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.482191 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:13.482204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.483193 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.483196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:13.483288 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.483406 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 18:43:13.483490 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:43:13.483618 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":292,"spinlock_wait_cycles":9856,"thread_start_us":68,"threads_started":1}
I20260929 18:43:13.484530 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b0f3ad4866487c8c0bc113129268ad" candidate_uuid: "4f971017b6af46fda779b6da2f72b85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1ae367c05d4a398733d23182cff608" is_pre_election: true
I20260929 18:43:13.484568 26334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b0f3ad4866487c8c0bc113129268ad" candidate_uuid: "4f971017b6af46fda779b6da2f72b85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe99c390d32475997cf7c1b9432d0c6" is_pre_election: true
I20260929 18:43:13.484876 26466 raft_consensus.cc:2454] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f971017b6af46fda779b6da2f72b85f in term 0.
I20260929 18:43:13.484876 26334 raft_consensus.cc:2454] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f971017b6af46fda779b6da2f72b85f in term 0.
I20260929 18:43:13.485383 26131 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f, 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6; no voters:
I20260929 18:43:13.485646 26248 raft_consensus.cc:2781] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:13.485728 26248 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:13.485777 26248 raft_consensus.cc:3037] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:13.486909 26248 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.487253 26248 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 1 election: Requested vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
I20260929 18:43:13.487589 26334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b0f3ad4866487c8c0bc113129268ad" candidate_uuid: "4f971017b6af46fda779b6da2f72b85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe99c390d32475997cf7c1b9432d0c6"
I20260929 18:43:13.487592 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b0f3ad4866487c8c0bc113129268ad" candidate_uuid: "4f971017b6af46fda779b6da2f72b85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1ae367c05d4a398733d23182cff608"
I20260929 18:43:13.487718 26334 raft_consensus.cc:3037] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:13.487718 26466 raft_consensus.cc:3037] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:13.489012 26466 raft_consensus.cc:2454] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f971017b6af46fda779b6da2f72b85f in term 1.
I20260929 18:43:13.489012 26334 raft_consensus.cc:2454] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f971017b6af46fda779b6da2f72b85f in term 1.
I20260929 18:43:13.489332 26131 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f, 5b1ae367c05d4a398733d23182cff608; no voters:
I20260929 18:43:13.489545 26248 raft_consensus.cc:2781] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:13.489722 26248 raft_consensus.cc:686] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 LEADER]: Becoming Leader. State: Replica: 4f971017b6af46fda779b6da2f72b85f, State: Running, Role: LEADER
I20260929 18:43:13.490000 26248 consensus_queue.cc:237] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:13.493173 26054 catalog_manager.cc:5495] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f971017b6af46fda779b6da2f72b85f (127.11.118.193). New cstate: current_term: 1 leader_uuid: "4f971017b6af46fda779b6da2f72b85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } health_report { overall_health: HEALTHY } } }
I20260929 18:43:13.515672 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.511843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.11.118.193:44585)
0929 18:43:13.512142 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:13.512148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:13.512159 (+ 11us) server_negotiation.cc:407] Connection header received
0929 18:43:13.512233 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:13.512235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:13.512296 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:13.512426 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:13.512683 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.513204 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.514090 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:13.514394 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:13.514505 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:13.514775 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:13.515252 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:13.515398 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:13.515441 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 18:43:13.515496 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260929 18:43:13.519577 26466 raft_consensus.cc:1261] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Refusing update from remote peer 4f971017b6af46fda779b6da2f72b85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:13.519577 26334 raft_consensus.cc:1261] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Refusing update from remote peer 4f971017b6af46fda779b6da2f72b85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:13.520356 26533 consensus_queue.cc:1035] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:13.520538 26248 consensus_queue.cc:1035] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:13.530237 26539 mvcc.cc:205] Tried to move back new op lower bound from 7334737483849740288 to 7334737483738058752. Current Snapshot: MvccSnapshot[applied={T|T < 7334737483849740288}]
I20260929 18:43:13.536063 26052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:13.536268 26052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:13.538786 26052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Sending DeleteTablet for 3 replicas of tablet e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:13.539680 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26092
I20260929 18:43:13.539608 26446 tablet_service.cc:1514] Processing DeleteTablet for tablet e2b0f3ad4866487c8c0bc113129268ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:13 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48494
I20260929 18:43:13.539593 26314 tablet_service.cc:1514] Processing DeleteTablet for tablet e2b0f3ad4866487c8c0bc113129268ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:13 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:60162
I20260929 18:43:13.540886 26547 tablet_replica.cc:330] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: stopping tablet replica
I20260929 18:43:13.541167 26547 raft_consensus.cc:2229] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:13.541517 26547 raft_consensus.cc:2258] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:13.541965 26548 tablet_replica.cc:330] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: stopping tablet replica
I20260929 18:43:13.542326 26548 raft_consensus.cc:2229] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:13.542632 26548 raft_consensus.cc:2258] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:13.543733 26547 ts_tablet_manager.cc:1905] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:13.543877 26548 ts_tablet_manager.cc:1905] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:13.546646 26548 ts_tablet_manager.cc:1918] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:13.546808 26548 log.cc:1196] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/e2b0f3ad4866487c8c0bc113129268ad
W20260929 18:43:13.546770 26040 connection.cc:535] client connection to 127.11.118.193:44585 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 18:43:13.546942 26040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 18:43:13.546990 26547 ts_tablet_manager.cc:1918] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:13.547078 26547 log.cc:1196] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:13.547240 26548 ts_tablet_manager.cc:1939] T e2b0f3ad4866487c8c0bc113129268ad P 5b1ae367c05d4a398733d23182cff608: Deleting consensus metadata
I20260929 18:43:13.547407 26547 ts_tablet_manager.cc:1939] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Deleting consensus metadata
I20260929 18:43:13.547467 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26246
I20260929 18:43:13.548432 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.547921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:44585 (local address 127.0.0.1:46118)
0929 18:43:13.548260 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.548340 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:44585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260929 18:43:13.548669 26041 catalog_manager.cc:4841] TS 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): tablet e2b0f3ad4866487c8c0bc113129268ad (table test-table [id=a40ec4f809c34be3989f4522331c9f59]) successfully deleted
W20260929 18:43:13.548615 26040 catalog_manager.cc:4584] TS 4f971017b6af46fda779b6da2f72b85f (127.11.118.193:44585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2b0f3ad4866487c8c0bc113129268ad: Network error: Client connection negotiation failed: client connection to 127.11.118.193:44585: connect: Connection refused (error 111)
I20260929 18:43:13.554075 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26383
I20260929 18:43:13.555053 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:13.554331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:41265 (local address 127.0.0.1:60166)
0929 18:43:13.554933 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.554978 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.194:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529}
I20260929 18:43:13.560184 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26021
I20260929 18:43:13.566817 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--webserver_interface=127.11.118.254
--webserver_port=45365
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:46165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:13.685259 26550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:13.685530 26550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:13.685607 26550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:13.689734 26550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:13.689800 26550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:13.689821 26550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:13.689877 26550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:13.689910 26550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:13.695469 26550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:46165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:46165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=45365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:13.696642 26550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:13.697705 26550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:13.706447 26558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:13.706522 26556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:13.706681 26555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.706517 26550 server_base.cc:1034] running on GCE node
I20260929 18:43:13.707284 26550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:13.708494 26550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:13.709694 26550 hybrid_clock.cc:648] HybridClock initialized: now 1790707393709660 us; error 49 us; skew 500 ppm
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:13.712563 26550 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:13.713492 26550 webserver.cc:460] Webserver started at http://127.11.118.254:45365/ using document root <none> and password file <none>
I20260929 18:43:13.714023 26550 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:13.714082 26550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:13.717800 26550 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:43:13.720155 26565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.721498 26550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:13.721644 26550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "41acffe7152d4f6ab5751c3d2233f4f1"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "0a71582d8ba585913712519f68c75807"
server_key_iv: "eb599d973f45d901b8ffdbc42bb2c337"
server_key_version: "encryptionkey@0"
I20260929 18:43:13.722254 26550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:13.739549 26550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:13.742928 26550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:13.743139 26550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:13.750550 26550 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:46165
I20260929 18:43:13.750859 26617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:46165 every 8 connection(s)
I20260929 18:43:13.751741 26550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:13.753607 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26550
I20260929 18:43:13.754057 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:44585
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=45069
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:13.757253 26618 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:43:13.759105 26618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap starting.
I20260929 18:43:13.776664 26618 log.cc:824] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:13.788354 26618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:13.788812 26618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: Bootstrap complete.
I20260929 18:43:13.792945 26618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:13.793363 26618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Initialized, Role: FOLLOWER
I20260929 18:43:13.793848 26618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:13.793987 26618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:13.794077 26618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:13.794193 26618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 2 FOLLOWER]: Advancing to term 3
I20260929 18:43:13.796097 26618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:13.796420 26618 leader_election.cc:302] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41acffe7152d4f6ab5751c3d2233f4f1; no voters:
I20260929 18:43:13.796761 26618 leader_election.cc:288] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 18:43:13.796875 26622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 18:43:13.797075 26622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [term 3 LEADER]: Becoming Leader. State: Replica: 41acffe7152d4f6ab5751c3d2233f4f1, State: Running, Role: LEADER
I20260929 18:43:13.797427 26622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } }
I20260929 18:43:13.797994 26618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:13.799191 26624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41acffe7152d4f6ab5751c3d2233f4f1. Latest consensus state: current_term: 3 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:13.799335 26624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:13.799587 26623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41acffe7152d4f6ab5751c3d2233f4f1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 46165 } } }
I20260929 18:43:13.799665 26623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:13.803478 26638 catalog_manager.cc:1259] Loaded cluster ID: cd759edd564f4bd6ba7fa75ad02e839e
I20260929 18:43:13.803539 26638 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: loading cluster ID for follower catalog manager: success
W20260929 18:43:13.804541 26638 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 41acffe7152d4f6ab5751c3d2233f4f1: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260929 18:43:13.804613 26638 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260929 18:43:13.804685 26629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:13.807040 26629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:13.807569 26629 catalog_manager.cc:1259] Loaded cluster ID: cd759edd564f4bd6ba7fa75ad02e839e
I20260929 18:43:13.807631 26629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 18:43:13.808285 26629 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa72bcd09ad google::LogMessage::Fail() at ??:0
@ 0x7fa72bcd388a google::LogMessage::SendToLog() at ??:0
@ 0x7fa72bcd04de google::LogMessage::Flush() at ??:0
@ 0x7fa72bcd1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa73660ffdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa736610d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa73660d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa736651640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa72df223d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa72df22c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa72df243f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa72df13512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa72f5576db start_thread at ??:0
@ 0x7fa72a7ce61f clone at ??:0
W20260929 18:43:13.895852 26620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:13.896147 26620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:13.896193 26620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:13.900604 26620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:13.900677 26620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:13.900776 26620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:13.906858 26620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=45069
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:13.908002 26620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:13.909065 26620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:13.918936 26660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:13.918936 26659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.918994 26620 server_base.cc:1034] running on GCE node
W20260929 18:43:13.918941 26662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:13.919617 26620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:13.920358 26620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:13.921561 26620 hybrid_clock.cc:648] HybridClock initialized: now 1790707393921544 us; error 44 us; skew 500 ppm
Sep 29 18:43:13 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:13.924527 26620 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:13.925516 26620 webserver.cc:460] Webserver started at http://127.11.118.193:45069/ using document root <none> and password file <none>
I20260929 18:43:13.926103 26620 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:13.926167 26620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:13.930063 26620 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:43:13.932549 26669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:13.934095 26620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:13.934197 26620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "4f971017b6af46fda779b6da2f72b85f"
format_stamp: "Formatted at 2026-09-29 18:43:11 on dist-test-slave-cgv7"
server_key: "64383def0c02c267b4f17db06b1742f2"
server_key_iv: "79b6969d94c2f63f30ad7b3023a80fde"
server_key_version: "encryptionkey@0"
I20260929 18:43:13.934926 26620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:13.950697 26620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:13.954584 26620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:13.954800 26620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:13.955533 26620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:13.956956 26676 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 18:43:13.961122 26620 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 18:43:13.961197 26620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260929 18:43:13.961262 26620 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 18:43:13.963537 26620 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 18:43:13.963595 26620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260929 18:43:13.963948 26676 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Bootstrap starting.
I20260929 18:43:13.974772 26620 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:44585
I20260929 18:43:13.974928 26783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:44585 every 8 connection(s)
I20260929 18:43:13.976099 26620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:13.982159 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26620
I20260929 18:43:13.982718 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:41265
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=37885
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:13.984318 26676 log.cc:824] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:13.994637 26676 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:13.995285 26676 tablet_bootstrap.cc:492] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Bootstrap complete.
I20260929 18:43:13.996824 26676 ts_tablet_manager.cc:1397] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.008s
I20260929 18:43:14.000191 26676 raft_consensus.cc:348] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:14.000806 26676 raft_consensus.cc:729] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f971017b6af46fda779b6da2f72b85f, State: Initialized, Role: FOLLOWER
I20260929 18:43:14.001433 26676 consensus_queue.cc:260] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:14.002599 26676 ts_tablet_manager.cc:1428] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
W20260929 18:43:14.054344 26786 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:43:13.978529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:46065)
0929 18:43:13.979144 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:13.979221 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:13.980374 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:14.053349 (+ 72975us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":186,"threads_started":1}
W20260929 18:43:14.054970 26784 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46165 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:46165: Client connection negotiation failed: client connection to 127.11.118.254:46165: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 18:43:14.055711 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.055431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:40621)
0929 18:43:14.055571 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.055645 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 18:43:14.056381 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.056141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:41779)
0929 18:43:14.056275 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.056325 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 18:43:14.056576 26784 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:43:14.105460 26787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:14.105737 26787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:14.105783 26787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:14.109946 26787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:14.110018 26787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:14.110122 26787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:14.115743 26787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:41265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=37885
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:14.116834 26787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:14.117875 26787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:14.127696 26794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:14.127707 26797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:14.127722 26787 server_base.cc:1034] running on GCE node
W20260929 18:43:14.128067 26795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:14.128520 26787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:14.129207 26787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:14.130400 26787 hybrid_clock.cc:648] HybridClock initialized: now 1790707394130388 us; error 27 us; skew 500 ppm
Sep 29 18:43:14 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707394, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:14.133169 26787 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:14.134096 26787 webserver.cc:460] Webserver started at http://127.11.118.194:37885/ using document root <none> and password file <none>
I20260929 18:43:14.134716 26787 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:14.134847 26787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:14.138670 26787 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260929 18:43:14.140944 26804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:14.142398 26787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:14.142530 26787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "dfe99c390d32475997cf7c1b9432d0c6"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "11b56ea7956d90d07d19efe0acdb9835"
server_key_iv: "a579847d36587569d69ea14790086ef8"
server_key_version: "encryptionkey@0"
I20260929 18:43:14.143172 26787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:14.160833 26787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:14.164016 26787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:14.164237 26787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:14.164865 26787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:14.165953 26811 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260929 18:43:14.170117 26811 ts_tablet_manager.cc:1852] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 18:43:14.170632 26811 ts_tablet_manager.cc:1905] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:14.174829 26811 ts_tablet_manager.cc:1918] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:14.175020 26811 ts_tablet_manager.cc:1939] T e2b0f3ad4866487c8c0bc113129268ad P dfe99c390d32475997cf7c1b9432d0c6: Deleting consensus metadata
I20260929 18:43:14.175493 26787 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 18:43:14.175599 26787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260929 18:43:14.175696 26787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:14.175750 26787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:14.184975 26787 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:41265
I20260929 18:43:14.185009 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:41265 every 8 connection(s)
I20260929 18:43:14.185948 26787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:14.189378 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.188096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:52875)
0929 18:43:14.188568 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.188678 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
W20260929 18:43:14.189798 26919 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46165 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:46165: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
I20260929 18:43:14.190461 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.190203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:53217)
0929 18:43:14.190349 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.190408 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 18:43:14.191268 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.191029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:43823)
0929 18:43:14.191164 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.191217 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 18:43:14.191529 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26787
W20260929 18:43:14.191546 26919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 18:43:14.191979 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:34243
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:46165
--webserver_port=44253
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:14.304885 26922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:14.305161 26922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:14.305207 26922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:14.309453 26922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:14.309525 26922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:14.309624 26922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:14.315137 26922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:34243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=44253
--tserver_master_addrs=127.11.118.254:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:14.316234 26922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:14.317190 26922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:14.326885 26928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:14.326920 26922 server_base.cc:1034] running on GCE node
W20260929 18:43:14.326876 26927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:14.326879 26930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:14.327564 26922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:14.328241 26922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:14.329411 26922 hybrid_clock.cc:648] HybridClock initialized: now 1790707394329378 us; error 45 us; skew 500 ppm
Sep 29 18:43:14 dist-test-slave-cgv7 krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707394, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:14.332285 26922 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:14.333140 26922 webserver.cc:460] Webserver started at http://127.11.118.195:44253/ using document root <none> and password file <none>
I20260929 18:43:14.333681 26922 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:14.333741 26922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:14.337734 26922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:14.340119 26937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:14.341457 26922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:43:14.341590 26922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "5b1ae367c05d4a398733d23182cff608"
format_stamp: "Formatted at 2026-09-29 18:43:12 on dist-test-slave-cgv7"
server_key: "2c92ea2570edc412c49ea7ba7c536221"
server_key_iv: "c0612d5e30f35096364883135d2d9dbe"
server_key_version: "encryptionkey@0"
I20260929 18:43:14.342177 26922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:14.359668 26922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:14.363129 26922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:14.363363 26922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:14.363994 26922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:14.364846 26922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:14.364892 26922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:14.364956 26922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:14.364991 26922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:14.374495 26922 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:34243
I20260929 18:43:14.374517 27050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:34243 every 8 connection(s)
I20260929 18:43:14.375521 26922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:14.378834 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 26922
I20260929 18:43:14.379698 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.377756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:36701)
0929 18:43:14.378262 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.378708 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":122,"threads_started":1}
W20260929 18:43:14.380195 27051 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:46165 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:46165: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
I20260929 18:43:14.380764 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.380542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:55621)
0929 18:43:14.380651 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.380706 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 18:43:14.381364 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:14.381160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:54347)
0929 18:43:14.381260 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:14.381305 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 18:43:14.381528 27051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 18:43:15.057752 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.057273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:41237)
0929 18:43:15.057568 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:15.057631 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260929 18:43:15.192186 27056 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:15.192420 27056 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:15.192811 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.192242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:38329)
0929 18:43:15.192534 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:15.192666 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260929 18:43:15.193619 27056 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
Sep 29 18:43:15 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.194@KRBTEST.COM
Sep 29 18:43:15 dist-test-slave-cgv7 krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707393, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.195@KRBTEST.COM
I20260929 18:43:15.207808 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.193890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:41265 (local address 127.11.118.193:58127)
0929 18:43:15.194389 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:15.194403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:15.194495 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:15.196072 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:15.196078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:15.196586 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:15.197120 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:15.197135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.198481 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.198486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:15.199466 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:15.199476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.200253 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.200256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:15.200379 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:15.201129 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:15.201140 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:15.202810 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:15.205120 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:15.205127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:15.205149 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:15.205400 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:15.205767 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:15.205770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:15.205772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:15.205905 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:15.206694 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:15.206701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:15.206983 (+ 282us) client_negotiation.cc:769] Sending connection context
0929 18:43:15.207179 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 18:43:15.207583 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":285,"spinlock_wait_cycles":259840,"thread_start_us":91,"threads_started":1}
I20260929 18:43:15.207923 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.194029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:58127 (local address 127.11.118.194:41265)
0929 18:43:15.194418 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:15.194429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:15.194467 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:43:15.195277 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:15.195314 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:15.195676 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:15.195979 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:15.197323 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.198341 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.199601 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.200087 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.200278 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:15.203057 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:15.203063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:15.203076 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:15.203105 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:15.204979 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:15.205524 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:15.205533 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:15.205543 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:15.205660 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:15.206004 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:15.206008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:15.206010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:15.206567 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:15.206727 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:15.207430 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 18:43:15.207719 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":60,"threads_started":1}
I20260929 18:43:15.208137 27060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.194054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:45869 (local address 127.11.118.195:34243)
0929 18:43:15.194434 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:15.194450 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:15.194494 (+ 44us) server_negotiation.cc:407] Connection header received
0929 18:43:15.195167 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:15.195172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:15.195594 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:15.196153 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:15.197330 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.198366 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.199606 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.200086 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.200257 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:15.203449 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:15.203457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:15.203474 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:15.203511 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:15.205346 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:15.205828 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:15.205833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:15.205837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:15.205950 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:15.206242 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:15.206247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:15.206248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:15.206773 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:15.207464 (+ 691us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:15.207613 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 18:43:15.207908 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":88,"threads_started":1}
I20260929 18:43:15.207919 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.193890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34243 (local address 127.11.118.193:45869)
0929 18:43:15.194019 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:15.194036 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:15.194146 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:15.195994 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:15.196000 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:15.196580 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:15.197119 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:15.197135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.198531 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.198533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:15.199468 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:15.199478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:15.200214 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:15.200218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:15.200382 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:15.201069 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:15.201084 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:15.203184 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:15.205474 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:15.205481 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:15.205483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:15.205688 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:15.206053 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:15.206057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:15.206059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:15.206146 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:15.206910 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:15.206913 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:15.207015 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 18:43:15.207173 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 18:43:15.207676 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":338,"spinlock_wait_cycles":7936}
W20260929 18:43:15.211087 26671 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
W20260929 18:43:15.211278 26673 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:15.211401 26673 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f; no voters: 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6
I20260929 18:43:15.211686 27056 raft_consensus.cc:2726] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:43:15.382644 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:15.382154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:41425)
0929 18:43:15.382482 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:15.382543 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":158,"threads_started":1}
I20260929 18:43:16.059048 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:16.058650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:36241)
0929 18:43:16.058901 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:16.058954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260929 18:43:16.194140 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:16.193655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:48747)
0929 18:43:16.193989 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:16.194050 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":123,"threads_started":1}
I20260929 18:43:16.384032 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:16.383551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:41187)
0929 18:43:16.383855 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:16.383920 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":75,"threads_started":1}
I20260929 18:43:17.060385 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:17.059901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:46935)
0929 18:43:17.060207 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:17.060273 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260929 18:43:17.153548 27066 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:17.153749 27066 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:17.154233 27066 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
W20260929 18:43:17.155108 26671 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
W20260929 18:43:17.155210 26673 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:17.155316 26673 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f; no voters: 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6
I20260929 18:43:17.155468 27066 raft_consensus.cc:2726] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:43:17.195796 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:17.195280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:49365)
0929 18:43:17.195570 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:17.195627 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260929 18:43:17.385293 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:17.384847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:43849)
0929 18:43:17.385122 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:17.385178 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260929 18:43:18.061648 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:18.061222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:49859)
0929 18:43:18.061485 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:18.061542 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260929 18:43:18.197074 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:18.196645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:41601)
0929 18:43:18.196915 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:18.196979 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260929 18:43:18.386650 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:18.386181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:40531)
0929 18:43:18.386492 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:18.386551 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":146,"threads_started":1}
I20260929 18:43:19.063174 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:19.062689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:48731)
0929 18:43:19.063024 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:19.063082 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":74,"threads_started":1}
I20260929 18:43:19.198477 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:19.197982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:53317)
0929 18:43:19.198252 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:19.198379 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260929 18:43:19.387984 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:19.387532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:35191)
0929 18:43:19.387816 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:19.387885 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260929 18:43:20.064636 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:20.064144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:59539)
0929 18:43:20.064410 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:20.064474 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260929 18:43:20.199733 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:20.199322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:54891)
0929 18:43:20.199577 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:20.199633 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":72,"threads_started":1}
I20260929 18:43:20.389485 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:20.388876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:45799)
0929 18:43:20.389310 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:20.389385 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":77,"threads_started":1}
I20260929 18:43:20.473954 27078 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:20.474121 27078 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:20.474771 27078 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
W20260929 18:43:20.475476 26671 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
W20260929 18:43:20.475579 26673 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:20.475689 26673 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f; no voters: 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6
I20260929 18:43:20.475840 27078 raft_consensus.cc:2726] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:43:21.066037 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:21.065574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:48235)
0929 18:43:21.065880 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:21.065942 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":107,"threads_started":1}
I20260929 18:43:21.201036 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:21.200543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:55599)
0929 18:43:21.200876 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:21.200941 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260929 18:43:21.390995 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:21.390470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:41143)
0929 18:43:21.390843 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:21.390902 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":86,"threads_started":1}
I20260929 18:43:22.067366 27082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:22.066951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:33381)
0929 18:43:22.067199 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:22.067270 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260929 18:43:22.202232 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:22.201805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:44847)
0929 18:43:22.202079 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:22.202136 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260929 18:43:22.392376 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:22.391821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:52931)
0929 18:43:22.392201 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:22.392267 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":108,"threads_started":1}
I20260929 18:43:23.068492 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:23.068105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:46513)
0929 18:43:23.068340 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:23.068396 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260929 18:43:23.203511 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:23.203061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:40935)
0929 18:43:23.203351 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:23.203414 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260929 18:43:23.393858 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:23.393270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:32987)
0929 18:43:23.393591 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:23.393658 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":86,"threads_started":1}
I20260929 18:43:24.069888 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:24.069429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:32827)
0929 18:43:24.069724 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:24.069790 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
I20260929 18:43:24.204968 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:24.204494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:40269)
0929 18:43:24.204806 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:24.204872 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":88,"threads_started":1}
I20260929 18:43:24.395147 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:24.394733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:51405)
0929 18:43:24.394995 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:24.395051 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260929 18:43:24.776731 27091 raft_consensus.cc:482] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:24.776932 27091 raft_consensus.cc:504] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe99c390d32475997cf7c1b9432d0c6" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 41265 } } peers { permanent_uuid: "5b1ae367c05d4a398733d23182cff608" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34243 } } peers { permanent_uuid: "4f971017b6af46fda779b6da2f72b85f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 44585 } }
I20260929 18:43:24.777433 27091 leader_election.cc:288] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265), 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243)
W20260929 18:43:24.778263 26671 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe99c390d32475997cf7c1b9432d0c6 (127.11.118.194:41265): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
W20260929 18:43:24.778420 26673 leader_election.cc:341] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5b1ae367c05d4a398733d23182cff608 (127.11.118.195:34243): Not found: Tablet not found: e2b0f3ad4866487c8c0bc113129268ad
I20260929 18:43:24.778530 26673 leader_election.cc:302] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f971017b6af46fda779b6da2f72b85f; no voters: 5b1ae367c05d4a398733d23182cff608, dfe99c390d32475997cf7c1b9432d0c6
I20260929 18:43:24.778698 27091 raft_consensus.cc:2726] T e2b0f3ad4866487c8c0bc113129268ad P 4f971017b6af46fda779b6da2f72b85f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:43:25.071230 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:25.070773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:45409)
0929 18:43:25.071074 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:25.071131 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260929 18:43:25.206208 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:25.205755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:53149)
0929 18:43:25.206044 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:25.206107 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260929 18:43:25.396605 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:25.396066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:50153)
0929 18:43:25.396445 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:25.396505 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":85,"threads_started":1}
I20260929 18:43:26.072449 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:26.072025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:58001)
0929 18:43:26.072290 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:26.072351 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260929 18:43:26.207418 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:26.207028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:51961)
0929 18:43:26.207267 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:26.207324 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
I20260929 18:43:26.398161 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:26.397665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:51751)
0929 18:43:26.397972 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:26.398060 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260929 18:43:27.073784 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:27.073307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:50055)
0929 18:43:27.073610 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:27.073686 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260929 18:43:27.208817 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:27.208325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:54077)
0929 18:43:27.208653 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:27.208719 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260929 18:43:27.399621 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:27.399126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:55289)
0929 18:43:27.399427 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:27.399494 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
I20260929 18:43:28.075443 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:28.074912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:42633)
0929 18:43:28.075271 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:28.075338 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":101,"threads_started":1}
I20260929 18:43:28.210250 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:28.209760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:40203)
0929 18:43:28.210088 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:28.210155 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":85,"threads_started":1}
I20260929 18:43:28.401010 27103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:28.400496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.195:34559)
0929 18:43:28.400842 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:28.400904 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260929 18:43:29.076958 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:29.076563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.193:44299)
0929 18:43:29.076808 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:29.076864 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260929 18:43:29.211814 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:29.211355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:46165 (local address 127.11.118.194:46411)
0929 18:43:29.211656 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:29.211720 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
I20260929 18:43:29.385492 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26620
I20260929 18:43:29.391367 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26787
I20260929 18:43:29.396965 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 26922
2026-09-29T18:43:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19185 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:43:29 dist-test-slave-cgv7 krb5kdc[27108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:43:29 dist-test-slave-cgv7 krb5kdc[27108](info): set up 2 sockets
Sep 29 18:43:29 dist-test-slave-cgv7 krb5kdc[27108](info): commencing operation
krb5kdc: starting...
W20260929 18:43:30.454711 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:43:30 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:43:30.587024 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:30.702734 27121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:30.703078 27121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:30.703159 27121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:30.707407 27121 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 18:43:30.707520 27121 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 18:43:30.707553 27121 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 18:43:30.707585 27121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:30.707620 27121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:30.707659 27121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:30.707698 27121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:30.707732 27121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:30.713653 27121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:36281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:30.714874 27121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:30.715970 27121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:30.725319 27129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:30.725234 27127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:30.725358 27121 server_base.cc:1034] running on GCE node
W20260929 18:43:30.725234 27126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:30.726246 27121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:30.727592 27121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:30.728826 27121 hybrid_clock.cc:648] HybridClock initialized: now 1790707410728787 us; error 58 us; skew 500 ppm
Sep 29 18:43:30 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707410, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:30.732206 27121 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:30.733278 27121 webserver.cc:460] Webserver started at http://127.11.118.254:36857/ using document root <none> and password file <none>
I20260929 18:43:30.733893 27121 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:30.734007 27121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:30.734262 27121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:30.736557 27121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "e5d2a34f41b746139b4141fe33864444"
format_stamp: "Formatted at 2026-09-29 18:43:30 on dist-test-slave-cgv7"
server_key: "7bbe4f6ab766ff6a471aa3ba522d3916"
server_key_iv: "b1b9e1b4b1c1d878769440cdbdfe3bde"
server_key_version: "encryptionkey@0"
I20260929 18:43:30.737195 27121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "e5d2a34f41b746139b4141fe33864444"
format_stamp: "Formatted at 2026-09-29 18:43:30 on dist-test-slave-cgv7"
server_key: "7bbe4f6ab766ff6a471aa3ba522d3916"
server_key_iv: "b1b9e1b4b1c1d878769440cdbdfe3bde"
server_key_version: "encryptionkey@0"
I20260929 18:43:30.741406 27121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:43:30.744489 27136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:30.745983 27121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 18:43:30.746207 27121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e5d2a34f41b746139b4141fe33864444"
format_stamp: "Formatted at 2026-09-29 18:43:30 on dist-test-slave-cgv7"
server_key: "7bbe4f6ab766ff6a471aa3ba522d3916"
server_key_iv: "b1b9e1b4b1c1d878769440cdbdfe3bde"
server_key_version: "encryptionkey@0"
I20260929 18:43:30.746451 27121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:30.759440 27121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:30.762838 27121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:30.763092 27121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:30.770414 27121 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:36281
I20260929 18:43:30.770792 27188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:36281 every 8 connection(s)
I20260929 18:43:30.771499 27121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:30.774351 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27121
I20260929 18:43:30.774518 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:43:30.774838 11739 external_mini_cluster.cc:1428] Setting key 519465409d4cd5406d3089907807133c
I20260929 18:43:30.775007 27189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:30.782608 27189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap starting.
Sep 29 18:43:30 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:30.785708 27189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:30.786756 27189 log.cc:824] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:30.788885 27189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: No bootstrap required, opened a new log
I20260929 18:43:30.791522 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:30.776782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.11.118.254:36281)
0929 18:43:30.777238 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:30.777254 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:30.777290 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:43:30.777960 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:30.777977 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:30.778395 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:30.778703 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:30.779834 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:30.781110 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:30.782394 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:30.783145 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:30.783280 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:30.786350 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:30.786358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:30.786377 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:30.786414 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:30.788352 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:30.788870 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:30.788880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:30.788886 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:30.789058 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:30.789382 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:30.789385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:30.789387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:30.789858 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:30.790096 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:30.790452 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 18:43:30.790781 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":172,"threads_started":1}
I20260929 18:43:30.793025 27189 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:30.793323 27189 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:30.793375 27189 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Initialized, Role: FOLLOWER
I20260929 18:43:30.793864 27189 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:30.794065 27189 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:30.794138 27189 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:30.794245 27189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:30.795318 27189 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:30.795642 27189 leader_election.cc:302] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5d2a34f41b746139b4141fe33864444; no voters:
I20260929 18:43:30.795924 27189 leader_election.cc:288] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:43:30.796070 27194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:30.796351 27194 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 LEADER]: Becoming Leader. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Running, Role: LEADER
I20260929 18:43:30.796762 27194 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:30.797257 27189 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:30.798713 27196 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5d2a34f41b746139b4141fe33864444. Latest consensus state: current_term: 1 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:43:30.798878 27196 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:30.798863 27195 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:43:30.798977 27195 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:30.799401 27203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:30.802934 27203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:30.808728 27203 catalog_manager.cc:1347] Generated new cluster ID: d60eeaf5c69746f8aa1a66d754821de9
I20260929 18:43:30.808820 27203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:30.826165 27203 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:43:30.827209 27203 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:30.847858 27203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Generated new TSK 0
I20260929 18:43:30.848719 27203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:43:30.883365 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:30.999410 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:30.999702 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:30.999747 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:31.003901 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:31.003971 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:31.004033 27215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:31.009483 27215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:31.010612 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:31.011615 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:31.021674 27220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:31.021658 27221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:31.021744 27215 server_base.cc:1034] running on GCE node
W20260929 18:43:31.021662 27223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:31.022468 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:31.023176 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:31.024371 27215 hybrid_clock.cc:648] HybridClock initialized: now 1790707411024341 us; error 49 us; skew 500 ppm
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:31.027276 27215 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:31.028216 27215 webserver.cc:460] Webserver started at http://127.11.118.193:37359/ using document root <none> and password file <none>
I20260929 18:43:31.028807 27215 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:31.028904 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:31.029146 27215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:31.031445 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "ed916a48cceb48a9a1efa2367e85416a"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "4b8ac661c0552dc0e89bcd514c8693a4"
server_key_iv: "65e4e11257f6f8163fab4fc7318dd25a"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.032035 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "ed916a48cceb48a9a1efa2367e85416a"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "4b8ac661c0552dc0e89bcd514c8693a4"
server_key_iv: "65e4e11257f6f8163fab4fc7318dd25a"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.035948 27215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 18:43:31.038477 27230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.039690 27215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:43:31.039830 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "ed916a48cceb48a9a1efa2367e85416a"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "4b8ac661c0552dc0e89bcd514c8693a4"
server_key_iv: "65e4e11257f6f8163fab4fc7318dd25a"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.039960 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:31.065308 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:31.068650 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:31.068868 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:31.069487 27215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:31.070441 27215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:31.070515 27215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.070582 27215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:31.070639 27215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.080037 27215 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45327
I20260929 18:43:31.080062 27343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45327 every 8 connection(s)
I20260929 18:43:31.080969 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:31.083114 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27215
I20260929 18:43:31.083256 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:43:31.083549 11739 external_mini_cluster.cc:1428] Setting key 61a0ec4bea7f07eac2b1e77b66acb98e
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:43:31.097683 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.082874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39817 (local address 127.11.118.254:36281)
0929 18:43:31.083026 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.083031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.084114 (+ 1083us) server_negotiation.cc:407] Connection header received
0929 18:43:31.085614 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.085618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.085678 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.085768 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:31.087291 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.088184 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.090089 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.090427 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.090515 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.093541 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:31.093545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:31.093547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:31.093572 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:31.095299 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.095890 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.095895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.095897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.095986 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.096376 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.096379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.096380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.096606 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:31.096726 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.097354 (+ 628us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.097545 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 18:43:31.098918 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.083330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:39817)
0929 18:43:31.083942 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.084017 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.085374 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.085927 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.085934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.086339 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:31.087096 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.087117 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.088373 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.088379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.089938 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.089955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.090608 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.090615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.090872 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.091553 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:31.091580 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:31.093344 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:31.095467 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.095474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.095491 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.095776 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.096088 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.096092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.096094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.096248 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.096725 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:31.096732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:31.097068 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.097310 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.097717 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":172,"threads_started":1}
I20260929 18:43:31.100323 27344 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:31.100603 27344 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:31.101140 27344 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:31.102828 27153 ts_manager.cc:194] Registered new tserver with Master: ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327)
I20260929 18:43:31.103953 27153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:39817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:43:31.117563 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:31.229274 27349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:31.229606 27349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:31.229689 27349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:31.233747 27349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:31.233848 27349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:31.233942 27349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:31.239177 27349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:31.240300 27349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:31.241430 27349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:31.251307 27357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:31.251307 27354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:31.251325 27355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:31.251679 27349 server_base.cc:1034] running on GCE node
I20260929 18:43:31.252056 27349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:31.252660 27349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:31.253834 27349 hybrid_clock.cc:648] HybridClock initialized: now 1790707411253817 us; error 27 us; skew 500 ppm
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:31.256655 27349 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:31.257475 27349 webserver.cc:460] Webserver started at http://127.11.118.194:46123/ using document root <none> and password file <none>
I20260929 18:43:31.257982 27349 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:31.258039 27349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:31.258208 27349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:31.260236 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "b4ff265e2f365ad707a448a71121ffb5"
server_key_iv: "394c553de2268f278ca5490a844e190b"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.260735 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "b4ff265e2f365ad707a448a71121ffb5"
server_key_iv: "394c553de2268f278ca5490a844e190b"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.264485 27349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:43:31.266932 27364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.268263 27349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:43:31.268464 27349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "b4ff265e2f365ad707a448a71121ffb5"
server_key_iv: "394c553de2268f278ca5490a844e190b"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.268625 27349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:31.284904 27349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:31.288564 27349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:31.288817 27349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:31.289446 27349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:31.290434 27349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:31.290510 27349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.290598 27349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:31.290632 27349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.300109 27349 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:42461
I20260929 18:43:31.300160 27477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:42461 every 8 connection(s)
I20260929 18:43:31.301064 27349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:31.304231 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27349
I20260929 18:43:31.304364 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:43:31.304677 11739 external_mini_cluster.cc:1428] Setting key 9ed50c74051c70fd2d8e628d3b0bd59f
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:43:31.318017 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.303234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:47567 (local address 127.11.118.254:36281)
0929 18:43:31.303420 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.303425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.304334 (+ 909us) server_negotiation.cc:407] Connection header received
0929 18:43:31.306049 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.306053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.306108 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.306191 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:31.307463 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.308189 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.310078 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.310392 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.310515 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.313746 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:31.313752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:31.313756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:31.313791 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:31.315650 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.316197 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.316199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.316200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.316327 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.316781 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.316783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.316784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.316967 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:31.317049 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.317720 (+ 671us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.317836 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:43:31.319012 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.303691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:47567)
0929 18:43:31.304169 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.304212 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.305747 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.306430 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.306437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.306800 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:31.307304 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.307318 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.308328 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.308333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.309934 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.309949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.310591 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.310599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.310854 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.311539 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:31.311567 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:31.313536 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:31.315786 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.315792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.315807 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.316096 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.316468 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.316475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.316480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.316681 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.317087 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:31.317092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:31.317438 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.317640 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.317925 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":104,"threads_started":1}
I20260929 18:43:31.320839 27478 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:31.321131 27478 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:31.321774 27478 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:31.322980 27153 ts_manager.cc:194] Registered new tserver with Master: 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
I20260929 18:43:31.323586 27153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:47567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:43:31.337811 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:31.452380 27483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:31.452679 27483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:31.452724 27483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:31.457412 27483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:31.457485 27483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:31.457602 27483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:31.463186 27483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:31.464380 27483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:31.465422 27483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:31.475626 27488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:31.475651 27489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:31.475678 27483 server_base.cc:1034] running on GCE node
W20260929 18:43:31.475636 27491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:31.476300 27483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:31.476976 27483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:31.478175 27483 hybrid_clock.cc:648] HybridClock initialized: now 1790707411478139 us; error 49 us; skew 500 ppm
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:31.481079 27483 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:31.481943 27483 webserver.cc:460] Webserver started at http://127.11.118.195:36513/ using document root <none> and password file <none>
I20260929 18:43:31.482558 27483 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:31.482625 27483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:31.482793 27483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:43:31.484931 27483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "2a51bf9019293d98e9c27886017ae7d8"
server_key_iv: "9d56d2a3278dd66f795e96193478cfcc"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.485523 27483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "2a51bf9019293d98e9c27886017ae7d8"
server_key_iv: "9d56d2a3278dd66f795e96193478cfcc"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.489504 27483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:43:31.492182 27498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.493511 27483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:31.493649 27483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "2a51bf9019293d98e9c27886017ae7d8"
server_key_iv: "9d56d2a3278dd66f795e96193478cfcc"
server_key_version: "encryptionkey@0"
I20260929 18:43:31.493772 27483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:31.517834 27483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:31.521705 27483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:31.521924 27483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:31.522596 27483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:31.523449 27483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:31.523526 27483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.523625 27483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:31.523674 27483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:31.532411 27483 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:41437
I20260929 18:43:31.532464 27611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:41437 every 8 connection(s)
I20260929 18:43:31.533416 27483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:31.535176 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27483
I20260929 18:43:31.535326 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:43:31.535646 11739 external_mini_cluster.cc:1428] Setting key 007b95ba330317b2c3e852ac2b50cdf2
Sep 29 18:43:31 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707411, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:31.547845 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.537604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:33469 (local address 127.11.118.254:36281)
0929 18:43:31.537768 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.537775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.537792 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:43:31.538462 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.538465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.538510 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.538579 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:31.539728 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.540258 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.541368 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.541609 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.541694 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.544329 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:31.544331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:31.544334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:31.544359 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:31.545751 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.546250 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.546253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.546254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.546410 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.546766 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.546769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.546771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.546979 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:31.547071 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.547607 (+ 536us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.547716 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:43:31.548820 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.536078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:33469)
0929 18:43:31.537104 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.537170 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.538178 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.538719 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.538726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.539078 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:31.539559 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.539572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.540363 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.540366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.541246 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.541254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.541742 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.541745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.541881 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.542572 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:31.542605 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:31.544168 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:31.545864 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.545869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.545883 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.546158 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.546498 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:31.546500 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:31.546502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:31.546648 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:31.547094 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:31.547099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:31.547350 (+ 251us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.547520 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.547797 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":612,"threads_started":1}
I20260929 18:43:31.550341 27612 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:31.550678 27612 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:31.551290 27612 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:31.552513 27152 ts_manager.cc:194] Registered new tserver with Master: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437)
I20260929 18:43:31.553171 27152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:33469
I20260929 18:43:31.562224 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:31.572309 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.564386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.11.118.254:36281)
0929 18:43:31.564579 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.564584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.564714 (+ 130us) server_negotiation.cc:407] Connection header received
0929 18:43:31.564878 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.564882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.564931 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.565026 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:31.565762 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.566355 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.567203 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.567425 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.567555 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.569095 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:31.569098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:31.569100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:31.569124 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:31.570442 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.570913 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.570919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.570921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.571007 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:31.571312 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:31.571315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:31.571317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:31.571557 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:31.571709 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.572006 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.572132 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260929 18:43:31.575812 27152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:43:31.578869 27152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:43:31.593806 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.588289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:42461 (local address 127.0.0.1:56208)
0929 18:43:31.588984 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.588998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.589096 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.589636 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.589639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.589663 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:31.589795 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.589799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.590901 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.590906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.592173 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.592183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.593202 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.593205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.593328 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.593370 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.593496 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.593631 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":4,"spinlock_wait_cycles":896,"thread_start_us":94,"threads_started":1}
I20260929 18:43:31.593812 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.588495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.11.118.194:42461)
0929 18:43:31.588793 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.588804 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.589110 (+ 306us) server_negotiation.cc:407] Connection header received
0929 18:43:31.589208 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.589214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.589406 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.589559 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:31.589913 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.590763 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.592313 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.593074 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.593201 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.593350 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.593551 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.593663 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260929 18:43:31.594774 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.587539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:41437 (local address 127.0.0.1:44908)
0929 18:43:31.587920 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.587981 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.588115 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.589569 (+ 1454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.589572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.589605 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:31.589755 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.589763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.591310 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.591313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.592821 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.592829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.594405 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.594409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.594498 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.594521 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.594568 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.594649 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":196,"thread_start_us":100,"threads_started":1}
I20260929 18:43:31.595293 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.587790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.11.118.193:45327)
0929 18:43:31.588160 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.588178 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.588507 (+ 329us) server_negotiation.cc:407] Connection header received
0929 18:43:31.589063 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.589071 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.589288 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.589453 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:31.589891 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.591171 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.592977 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.593873 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.593995 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.594114 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.595060 (+ 946us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.595181 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":79,"threads_started":1}
I20260929 18:43:31.595353 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.588707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.11.118.195:41437)
0929 18:43:31.589118 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.589137 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.589159 (+ 22us) server_negotiation.cc:407] Connection header received
0929 18:43:31.589222 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.589225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.589406 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.589742 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:31.589901 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.591145 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.592969 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.593855 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.594747 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.595065 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.595120 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.595214 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":177,"threads_started":1}
I20260929 18:43:31.594648 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.587564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45327 (local address 127.0.0.1:48312)
0929 18:43:31.588291 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.588304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.588392 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.589453 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.589459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.589483 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:31.589645 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.589653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.591310 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.591313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.592818 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.592829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.594171 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.594174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.594316 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.594340 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.594411 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.594504 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":133,"thread_start_us":96,"threads_started":1}
I20260929 18:43:31.597342 27412 tablet_service.cc:1467] Processing CreateTablet for tablet 61f9e638129244038981f365c1806778 (DEFAULT_TABLE table=test-table [id=56b73818ddc641b4aa6158f1dec959a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:31.598693 27412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f9e638129244038981f365c1806778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:31.598407 27546 tablet_service.cc:1467] Processing CreateTablet for tablet 61f9e638129244038981f365c1806778 (DEFAULT_TABLE table=test-table [id=56b73818ddc641b4aa6158f1dec959a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:31.599632 27546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f9e638129244038981f365c1806778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:31.600188 27278 tablet_service.cc:1467] Processing CreateTablet for tablet 61f9e638129244038981f365c1806778 (DEFAULT_TABLE table=test-table [id=56b73818ddc641b4aa6158f1dec959a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:31.601228 27278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f9e638129244038981f365c1806778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:31.605333 27629 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Bootstrap starting.
I20260929 18:43:31.607059 27630 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Bootstrap starting.
I20260929 18:43:31.608086 27631 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Bootstrap starting.
I20260929 18:43:31.609392 27629 tablet_bootstrap.cc:654] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:31.609436 27630 tablet_bootstrap.cc:654] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:31.610176 27630 log.cc:824] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:31.610302 27631 tablet_bootstrap.cc:654] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:31.610248 27629 log.cc:824] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:31.611313 27631 log.cc:824] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:31.612370 27630 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: No bootstrap required, opened a new log
I20260929 18:43:31.612370 27629 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: No bootstrap required, opened a new log
I20260929 18:43:31.612577 27630 ts_tablet_manager.cc:1397] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:43:31.612577 27629 ts_tablet_manager.cc:1397] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260929 18:43:31.613180 27631 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: No bootstrap required, opened a new log
I20260929 18:43:31.613390 27631 ts_tablet_manager.cc:1397] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:43:31.616704 27630 raft_consensus.cc:348] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.616861 27629 raft_consensus.cc:348] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.617046 27630 raft_consensus.cc:374] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:31.617079 27629 raft_consensus.cc:374] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:31.617122 27630 raft_consensus.cc:729] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, State: Initialized, Role: FOLLOWER
I20260929 18:43:31.617134 27629 raft_consensus.cc:729] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5ae1b80e9d4e5e9b844a135b915b07, State: Initialized, Role: FOLLOWER
I20260929 18:43:31.617233 27631 raft_consensus.cc:348] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.617568 27631 raft_consensus.cc:374] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:31.617632 27631 raft_consensus.cc:729] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed916a48cceb48a9a1efa2367e85416a, State: Initialized, Role: FOLLOWER
I20260929 18:43:31.617624 27630 consensus_queue.cc:260] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.617624 27629 consensus_queue.cc:260] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.618120 27631 consensus_queue.cc:260] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.618438 27478 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
I20260929 18:43:31.618731 27630 ts_tablet_manager.cc:1428] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260929 18:43:31.618731 27629 ts_tablet_manager.cc:1428] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260929 18:43:31.619067 27612 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
I20260929 18:43:31.619303 27631 ts_tablet_manager.cc:1428] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:43:31.619449 27344 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
W20260929 18:43:31.784430 27613 tablet.cc:2369] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:43:31.802477 27479 tablet.cc:2369] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:43:31.832382 27345 tablet.cc:2369] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:43:31.933692 27635 raft_consensus.cc:482] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:31.933962 27635 raft_consensus.cc:504] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.935263 27635 leader_election.cc:288] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327)
I20260929 18:43:31.939267 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.935741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:45593 (local address 127.11.118.195:41437)
0929 18:43:31.935902 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.935907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.935927 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:43:31.936105 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.936109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.936166 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.936269 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:31.936560 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.937078 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938166 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.938683 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938821 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.938874 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.939077 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.939140 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 18:43:31.939282 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.935592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:41437 (local address 127.11.118.194:45593)
0929 18:43:31.935861 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.935873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.935970 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.936281 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.936284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.936301 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:31.936430 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.936436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.937198 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.937205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.938030 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.938039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.938815 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.938915 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.938946 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.939056 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.939132 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260929 18:43:31.939374 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.935715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:60357 (local address 127.11.118.193:45327)
0929 18:43:31.935855 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.935860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.936216 (+ 356us) server_negotiation.cc:407] Connection header received
0929 18:43:31.936369 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.936372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.936424 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.936515 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:31.936779 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.937275 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938379 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.938805 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938939 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.938993 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.939179 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.939249 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:43:31.939412 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.935588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45327 (local address 127.11.118.194:60357)
0929 18:43:31.936110 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:31.936122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:31.936255 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:31.936522 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:31.936525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:31.936540 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:31.936668 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.936674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.937391 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.937396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.938232 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:31.938241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.938931 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.938934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:31.939045 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.939066 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 18:43:31.939165 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 18:43:31.939235 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":343,"thread_start_us":63,"threads_started":1}
I20260929 18:43:31.939924 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f9e638129244038981f365c1806778" candidate_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" is_pre_election: true
I20260929 18:43:31.940068 27298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f9e638129244038981f365c1806778" candidate_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed916a48cceb48a9a1efa2367e85416a" is_pre_election: true
I20260929 18:43:31.940296 27566 raft_consensus.cc:2454] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5ae1b80e9d4e5e9b844a135b915b07 in term 0.
I20260929 18:43:31.940367 27298 raft_consensus.cc:2454] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5ae1b80e9d4e5e9b844a135b915b07 in term 0.
I20260929 18:43:31.940811 27366 leader_election.cc:302] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07; no voters:
I20260929 18:43:31.941136 27635 raft_consensus.cc:2781] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:31.941233 27635 raft_consensus.cc:482] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:31.941275 27635 raft_consensus.cc:3037] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:31.942324 27635 raft_consensus.cc:504] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.942670 27635 leader_election.cc:288] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [CANDIDATE]: Term 1 election: Requested vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327)
I20260929 18:43:31.942978 27298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f9e638129244038981f365c1806778" candidate_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed916a48cceb48a9a1efa2367e85416a"
I20260929 18:43:31.943092 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f9e638129244038981f365c1806778" candidate_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
I20260929 18:43:31.943140 27298 raft_consensus.cc:3037] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:31.943265 27566 raft_consensus.cc:3037] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:31.944371 27298 raft_consensus.cc:2454] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5ae1b80e9d4e5e9b844a135b915b07 in term 1.
I20260929 18:43:31.944463 27566 raft_consensus.cc:2454] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5ae1b80e9d4e5e9b844a135b915b07 in term 1.
I20260929 18:43:31.944746 27367 leader_election.cc:302] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5ae1b80e9d4e5e9b844a135b915b07, ed916a48cceb48a9a1efa2367e85416a; no voters:
I20260929 18:43:31.944952 27635 raft_consensus.cc:2781] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:31.945209 27635 raft_consensus.cc:686] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 LEADER]: Becoming Leader. State: Replica: 6e5ae1b80e9d4e5e9b844a135b915b07, State: Running, Role: LEADER
I20260929 18:43:31.945564 27635 consensus_queue.cc:237] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:31.948791 27152 catalog_manager.cc:5495] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194). New cstate: current_term: 1 leader_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } health_report { overall_health: HEALTHY } } }
I20260929 18:43:31.963433 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:31.959917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.11.118.194:42461)
0929 18:43:31.960049 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:31.960052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:31.960065 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:43:31.960191 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:31.960193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:31.960249 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:31.960335 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:31.960608 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.961091 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.961916 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:31.962119 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:31.962192 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:31.962556 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:31.963047 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:31.963138 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:31.963209 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 18:43:31.963259 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:43:31.969326 27566 raft_consensus.cc:1261] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Refusing update from remote peer 6e5ae1b80e9d4e5e9b844a135b915b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:31.969373 27298 raft_consensus.cc:1261] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Refusing update from remote peer 6e5ae1b80e9d4e5e9b844a135b915b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:31.970018 27635 consensus_queue.cc:1035] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:31.970199 27640 consensus_queue.cc:1035] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:31.978762 27645 mvcc.cc:205] Tried to move back new op lower bound from 7334737559419260928 to 7334737559331643392. Current Snapshot: MvccSnapshot[applied={T|T < 7334737559419260928}]
I20260929 18:43:31.979080 27644 mvcc.cc:205] Tried to move back new op lower bound from 7334737559419260928 to 7334737559331643392. Current Snapshot: MvccSnapshot[applied={T|T < 7334737559419260928}]
I20260929 18:43:31.986088 27152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:31.986306 27152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:31.988890 27152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Sending DeleteTablet for 3 replicas of tablet 61f9e638129244038981f365c1806778
I20260929 18:43:31.989791 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27215
I20260929 18:43:31.989825 27278 tablet_service.cc:1514] Processing DeleteTablet for tablet 61f9e638129244038981f365c1806778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:48312
I20260929 18:43:31.990522 27546 tablet_service.cc:1514] Processing DeleteTablet for tablet 61f9e638129244038981f365c1806778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:44908
I20260929 18:43:31.991021 27652 tablet_replica.cc:330] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: stopping tablet replica
I20260929 18:43:31.991204 27412 tablet_service.cc:1514] Processing DeleteTablet for tablet 61f9e638129244038981f365c1806778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:56208
I20260929 18:43:31.991521 27652 raft_consensus.cc:2229] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:31.991820 27653 tablet_replica.cc:330] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: stopping tablet replica
I20260929 18:43:31.991957 27652 raft_consensus.cc:2258] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:31.992100 27653 raft_consensus.cc:2229] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:43:31.992473 27653 raft_consensus.cc:2258] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:31.993779 27653 ts_tablet_manager.cc:1905] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:31.993779 27652 ts_tablet_manager.cc:1905] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:31.996567 27652 ts_tablet_manager.cc:1918] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:31.996583 27653 ts_tablet_manager.cc:1918] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:31.996671 27653 log.cc:1196] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/61f9e638129244038981f365c1806778
I20260929 18:43:31.996794 27652 log.cc:1196] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/61f9e638129244038981f365c1806778
I20260929 18:43:31.996996 27653 ts_tablet_manager.cc:1939] T 61f9e638129244038981f365c1806778 P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting consensus metadata
I20260929 18:43:31.997237 27652 ts_tablet_manager.cc:1939] T 61f9e638129244038981f365c1806778 P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting consensus metadata
I20260929 18:43:31.998423 27138 catalog_manager.cc:4841] TS 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): tablet 61f9e638129244038981f365c1806778 (table test-table [id=56b73818ddc641b4aa6158f1dec959a6]) successfully deleted
I20260929 18:43:31.998729 27139 catalog_manager.cc:4841] TS 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): tablet 61f9e638129244038981f365c1806778 (table test-table [id=56b73818ddc641b4aa6158f1dec959a6]) successfully deleted
W20260929 18:43:31.999428 27139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:45327 (error 108)
I20260929 18:43:31.999660 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27349
I20260929 18:43:32.001374 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:32.001085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45327 (local address 127.0.0.1:48318)
0929 18:43:32.001229 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:32.001300 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 18:43:32.001657 27139 catalog_manager.cc:4584] TS ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61f9e638129244038981f365c1806778: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45327: connect: Connection refused (error 111)
I20260929 18:43:32.005625 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27483
I20260929 18:43:32.011806 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27121
I20260929 18:43:52.018674 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--webserver_interface=127.11.118.254
--webserver_port=36857
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:43:52.131901 27655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:52.132195 27655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:52.132238 27655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:52.136394 27655 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 18:43:52.136458 27655 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 18:43:52.136474 27655 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 18:43:52.136489 27655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:43:52.136508 27655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:52.136528 27655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:52.136595 27655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:43:52.136623 27655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:43:52.143630 27655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:36281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:52.144831 27655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:52.145934 27655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:52.160004 27662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.159900 27664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.160012 27655 server_base.cc:1034] running on GCE node
W20260929 18:43:52.159900 27661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.160887 27655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:52.162232 27655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:52.163477 27655 hybrid_clock.cc:648] HybridClock initialized: now 1790707432163433 us; error 60 us; skew 500 ppm
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:52.172981 27655 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:43:52.174125 27655 webserver.cc:460] Webserver started at http://127.11.118.254:36857/ using document root <none> and password file <none>
I20260929 18:43:52.174863 27655 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:52.174988 27655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:52.179550 27655 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:43:52.181941 27671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.183522 27655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:52.183712 27655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e5d2a34f41b746139b4141fe33864444"
format_stamp: "Formatted at 2026-09-29 18:43:30 on dist-test-slave-cgv7"
server_key: "7bbe4f6ab766ff6a471aa3ba522d3916"
server_key_iv: "b1b9e1b4b1c1d878769440cdbdfe3bde"
server_key_version: "encryptionkey@0"
I20260929 18:43:52.184379 27655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:52.211727 27655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:52.215199 27655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:52.215436 27655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:52.223093 27655 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:36281
I20260929 18:43:52.223591 27723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:36281 every 8 connection(s)
I20260929 18:43:52.224292 27655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:43:52.226132 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27655
I20260929 18:43:52.226748 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:45327
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=37359
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:52.229092 27724 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:43:52.230896 27724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap starting.
I20260929 18:43:52.248136 27724 log.cc:824] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:52.256925 27724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:52.257359 27724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap complete.
I20260929 18:43:52.261488 27724 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:52.261901 27724 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Initialized, Role: FOLLOWER
I20260929 18:43:52.262463 27724 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:52.262655 27724 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:43:52.262732 27724 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:43:52.262853 27724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 1 FOLLOWER]: Advancing to term 2
I20260929 18:43:52.264899 27724 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:52.265250 27724 leader_election.cc:302] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5d2a34f41b746139b4141fe33864444; no voters:
I20260929 18:43:52.265591 27724 leader_election.cc:288] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 18:43:52.265710 27728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 18:43:52.265929 27728 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 LEADER]: Becoming Leader. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Running, Role: LEADER
I20260929 18:43:52.266335 27728 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:43:52.266952 27724 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:43:52.267974 27730 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5d2a34f41b746139b4141fe33864444. Latest consensus state: current_term: 2 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:43:52.268111 27730 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:52.268481 27729 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:43:52.268567 27729 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:43:52.272066 27744 catalog_manager.cc:1259] Loaded cluster ID: d60eeaf5c69746f8aa1a66d754821de9
I20260929 18:43:52.272152 27744 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: loading cluster ID for follower catalog manager: success
I20260929 18:43:52.273904 27744 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: acquiring CA information for follower catalog manager: success
I20260929 18:43:52.275184 27744 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:43:52.275344 27737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:43:52.277321 27737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:43:52.277767 27737 catalog_manager.cc:1259] Loaded cluster ID: d60eeaf5c69746f8aa1a66d754821de9
I20260929 18:43:52.277842 27737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:43:52.278502 27737 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:43:52.278919 27737 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Loaded TSK: 0
I20260929 18:43:52.297952 27737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Generated new TSK 1
I20260929 18:43:52.298245 27737 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:43:52.357633 27726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:52.357949 27726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:52.358026 27726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:52.362335 27726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:52.362430 27726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:52.362524 27726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:43:52.368549 27726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:45327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=37359
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:52.369796 27726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:52.371059 27726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:52.381918 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.381918 27754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.381928 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.382421 27726 server_base.cc:1034] running on GCE node
I20260929 18:43:52.382853 27726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:52.383512 27726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:52.384727 27726 hybrid_clock.cc:648] HybridClock initialized: now 1790707432384692 us; error 46 us; skew 500 ppm
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:52.387946 27726 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:43:52.389038 27726 webserver.cc:460] Webserver started at http://127.11.118.193:37359/ using document root <none> and password file <none>
I20260929 18:43:52.389599 27726 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:52.389672 27726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:52.393654 27726 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:52.395962 27761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.397279 27726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:43:52.397372 27726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "ed916a48cceb48a9a1efa2367e85416a"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "4b8ac661c0552dc0e89bcd514c8693a4"
server_key_iv: "65e4e11257f6f8163fab4fc7318dd25a"
server_key_version: "encryptionkey@0"
I20260929 18:43:52.397933 27726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:52.420605 27726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:52.424052 27726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:52.424258 27726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:52.424961 27726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:52.426108 27768 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 18:43:52.430351 27726 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 18:43:52.430434 27726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260929 18:43:52.430488 27726 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 18:43:52.432209 27726 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 18:43:52.432282 27726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 18:43:52.432538 27768 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Bootstrap starting.
I20260929 18:43:52.443583 27726 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45327
I20260929 18:43:52.443674 27875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45327 every 8 connection(s)
I20260929 18:43:52.444877 27726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:43:52.445107 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27726
I20260929 18:43:52.445626 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:42461
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=46123
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:43:52.455328 27768 log.cc:824] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Log is configured to *not* fsync() on all Append() calls
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:52.464038 27768 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:43:52.464473 27768 tablet_bootstrap.cc:492] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Bootstrap complete.
I20260929 18:43:52.465492 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.447129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:43707)
0929 18:43:52.447618 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.447670 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.448790 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.450309 (+ 1519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.450322 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.450881 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:52.451630 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.451655 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.453693 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.453700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.455338 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.455353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.456524 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.456528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.456714 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.457428 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:52.457448 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:52.459434 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:52.461970 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.461977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.461992 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.462369 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.462776 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.462780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.462782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.462950 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.463705 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:52.463718 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:52.464004 (+ 286us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.464183 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.464512 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":142,"threads_started":1}
I20260929 18:43:52.465605 27768 ts_tablet_manager.cc:1397] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.008s
I20260929 18:43:52.467217 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.447345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:43707 (local address 127.11.118.254:36281)
0929 18:43:52.448205 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.448235 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.448283 (+ 48us) server_negotiation.cc:407] Connection header received
0929 18:43:52.449318 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.449336 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.449769 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.450193 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.451884 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.453500 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.455508 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.456357 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.456556 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.459658 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.459672 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.459688 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.459722 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.461825 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.462501 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.462508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.462515 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.462678 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.463067 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.463071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.463072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.463522 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.463660 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.465988 (+ 2328us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.466358 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":128,"threads_started":1}
I20260929 18:43:52.468883 27768 raft_consensus.cc:348] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:52.469520 27768 raft_consensus.cc:729] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed916a48cceb48a9a1efa2367e85416a, State: Initialized, Role: FOLLOWER
I20260929 18:43:52.469821 27876 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:52.470026 27768 consensus_queue.cc:260] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } }
I20260929 18:43:52.470036 27876 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:52.470845 27876 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:52.471251 27768 ts_tablet_manager.cc:1428] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260929 18:43:52.473515 27688 ts_manager.cc:194] Registered new tserver with Master: ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327)
I20260929 18:43:52.477169 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:43707
I20260929 18:43:52.479977 27876 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.193@KRBTEST.COM
I20260929 18:43:52.487623 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.476572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.11.118.193:45327)
0929 18:43:52.477271 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.477285 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.477313 (+ 28us) server_negotiation.cc:407] Connection header received
0929 18:43:52.477390 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.477394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.477592 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.477738 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.478529 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.479573 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.480790 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.481311 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.481423 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.483930 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.483939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.483948 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.483970 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.485610 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.486162 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.486190 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.486192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.486322 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.486617 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.486619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.486620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.486856 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.486955 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.487335 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.487471 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":140,"threads_started":1}
I20260929 18:43:52.487636 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.476483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45327 (local address 127.0.0.1:57324)
0929 18:43:52.476957 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.477008 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.477142 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.477767 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.477772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.478212 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:52.478385 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.478395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.479704 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.479708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.480657 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.480669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.481441 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.481445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.481545 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.482102 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:52.482117 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:52.483781 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:52.485730 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.485739 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.485745 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.486046 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.486424 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.486428 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.486430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.486526 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.486977 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:52.486982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:52.487111 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.487259 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.487459 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":125,"threads_started":1}
I20260929 18:43:52.488798 27810 tablet_service.cc:1514] Processing DeleteTablet for tablet 61f9e638129244038981f365c1806778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57324
I20260929 18:43:52.489799 27885 tablet_replica.cc:330] stopping tablet replica
I20260929 18:43:52.490141 27885 raft_consensus.cc:2229] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:52.490432 27885 raft_consensus.cc:2258] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:52.491536 27885 ts_tablet_manager.cc:1905] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:52.496107 27885 ts_tablet_manager.cc:1918] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:52.496210 27885 log.cc:1196] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal/wals/61f9e638129244038981f365c1806778
I20260929 18:43:52.496533 27885 ts_tablet_manager.cc:1939] T 61f9e638129244038981f365c1806778 P ed916a48cceb48a9a1efa2367e85416a: Deleting consensus metadata
I20260929 18:43:52.498591 27674 catalog_manager.cc:4841] TS ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327): tablet 61f9e638129244038981f365c1806778 (table test-table [id=56b73818ddc641b4aa6158f1dec959a6]) successfully deleted
W20260929 18:43:52.574990 27878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:52.575639 27878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:52.575735 27878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:52.580431 27878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:52.580498 27878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:52.580595 27878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:43:52.586124 27878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:42461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=46123
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:52.587463 27878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:52.588548 27878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:52.598495 27891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.598495 27893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.598500 27890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.598843 27878 server_base.cc:1034] running on GCE node
I20260929 18:43:52.599267 27878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:52.599896 27878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:52.601082 27878 hybrid_clock.cc:648] HybridClock initialized: now 1790707432601069 us; error 28 us; skew 500 ppm
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:52.604278 27878 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:43:52.605170 27878 webserver.cc:460] Webserver started at http://127.11.118.194:46123/ using document root <none> and password file <none>
I20260929 18:43:52.605726 27878 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:52.605796 27878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:52.609740 27878 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:43:52.611955 27900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.613328 27878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:43:52.613461 27878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "b4ff265e2f365ad707a448a71121ffb5"
server_key_iv: "394c553de2268f278ca5490a844e190b"
server_key_version: "encryptionkey@0"
I20260929 18:43:52.614020 27878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:52.633889 27878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:52.637352 27878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:52.637575 27878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:52.638195 27878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:52.639075 27878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:52.639122 27878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.639160 27878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:52.639218 27878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.648104 27878 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:42461
I20260929 18:43:52.648183 28013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:42461 every 8 connection(s)
I20260929 18:43:52.649165 27878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:43:52.653488 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 27878
I20260929 18:43:52.654006 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:41437
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=36513
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:52.665094 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.651158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:42861 (local address 127.11.118.254:36281)
0929 18:43:52.651314 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.651317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.652055 (+ 738us) server_negotiation.cc:407] Connection header received
0929 18:43:52.653462 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.653465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.653531 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.653652 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.655452 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.656363 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.658084 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.658422 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.658550 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.661138 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.661142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.661144 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.661170 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.662787 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.663417 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.663420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.663422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.663508 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.663933 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.663938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.663940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.664142 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.664252 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.664776 (+ 524us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.664925 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:43:52.665973 28016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.651465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:42861)
0929 18:43:52.651902 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.651945 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.653238 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.653914 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.653934 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.654504 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:52.655244 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.655265 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.656513 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.656516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.657936 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.657951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.658595 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.658599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.658802 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.659428 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:52.659445 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:52.660938 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:52.662908 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.662915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.662934 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.663303 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.663602 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.663606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.663608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.663836 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.664249 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:52.664256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:52.664532 (+ 276us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.664737 (+ 205us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.665085 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260929 18:43:52.667420 28014 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:52.667735 28014 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:52.668489 28014 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:52.669754 27688 ts_manager.cc:194] Registered new tserver with Master: 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
I20260929 18:43:52.670583 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:42861
W20260929 18:43:52.773365 28017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:43:52.773639 28017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:43:52.773684 28017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:43:52.777776 28017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:43:52.777839 28017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:43:52.777901 28017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:43:52.783396 28017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:41437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=36513
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:43:52.784663 28017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:43:52.785770 28017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:43:52.796106 28025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:43:52.796125 28023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.796296 28017 server_base.cc:1034] running on GCE node
W20260929 18:43:52.796151 28022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:43:52.796877 28017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:43:52.797564 28017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:43:52.798718 28017 hybrid_clock.cc:648] HybridClock initialized: now 1790707432798703 us; error 29 us; skew 500 ppm
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:43:52.801674 28017 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:43:52.802683 28017 webserver.cc:460] Webserver started at http://127.11.118.195:36513/ using document root <none> and password file <none>
I20260929 18:43:52.803256 28017 fs_manager.cc:359] Metadata directory not provided
I20260929 18:43:52.803335 28017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:43:52.808277 28017 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:43:52.811458 28032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.813272 28017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:43:52.813465 28017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "2a51bf9019293d98e9c27886017ae7d8"
server_key_iv: "9d56d2a3278dd66f795e96193478cfcc"
server_key_version: "encryptionkey@0"
I20260929 18:43:52.814311 28017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:43:52.836624 28017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:43:52.840754 28017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:43:52.840967 28017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:43:52.841602 28017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:43:52.842548 28017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:43:52.842598 28017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.842639 28017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:43:52.842659 28017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:43:52.853430 28017 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:41437
I20260929 18:43:52.853487 28145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:41437 every 8 connection(s)
I20260929 18:43:52.854763 28017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:43:52.862358 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28017
Sep 29 18:43:52 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:43:52.870237 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.856767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:36261 (local address 127.11.118.254:36281)
0929 18:43:52.856909 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.856913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.857824 (+ 911us) server_negotiation.cc:407] Connection header received
0929 18:43:52.859279 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.859283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.859345 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.859426 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.860654 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.861260 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.862527 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.862755 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.862873 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.865819 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.865823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.865827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.865858 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.867697 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.868310 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.868313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.868315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.868396 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.868777 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.868780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.868783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.869003 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.869104 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.869878 (+ 774us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.870028 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 18:43:52.871177 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.857177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:36261)
0929 18:43:52.857665 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.857706 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.859053 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.859597 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.859604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.860007 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:43:52.860501 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.860515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.861396 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.861400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.862387 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.862396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.862912 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.862917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.863140 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.863797 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:43:52.863818 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:43:52.865674 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:43:52.867813 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.867819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.867837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.868183 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.868498 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:43:52.868501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:43:52.868503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:43:52.868688 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:43:52.869159 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:43:52.869168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:43:52.869546 (+ 378us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.869784 (+ 238us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.870120 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
I20260929 18:43:52.872421 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.863492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.11.118.254:36281)
0929 18:43:52.863854 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.863858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.863872 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:52.863910 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.863913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.863960 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.864079 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.864920 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.865612 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.866573 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.866963 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.867083 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.868819 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.868825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.868829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.868850 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.870589 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.871025 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.871031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.871033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.871115 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.871435 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.871439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.871440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.871612 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.871847 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.872149 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.872264 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":66,"threads_started":1}
I20260929 18:43:52.872581 28146 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36281
I20260929 18:43:52.872850 28146 heartbeater.cc:464] Registering TS with master...
I20260929 18:43:52.873452 28146 heartbeater.cc:511] Master 127.11.118.254:36281 requested a full tablet report, sending...
I20260929 18:43:52.874531 27688 ts_manager.cc:194] Registered new tserver with Master: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437)
I20260929 18:43:52.875078 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:36261
I20260929 18:43:52.886066 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:43:52.890615 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.886789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.11.118.254:36281)
0929 18:43:52.886919 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.886923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.887133 (+ 210us) server_negotiation.cc:407] Connection header received
0929 18:43:52.887290 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.887293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.887404 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.887493 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:52.887744 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.888356 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.889350 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.889576 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.889658 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.889976 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:52.890134 (+ 158us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 18:43:52.890506 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44008: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260929 18:43:52.890715 28150 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44008: token expired
W20260929 18:43:52.891052 27617 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.118.254:36281)
I20260929 18:43:52.898547 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.891447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.11.118.254:36281)
0929 18:43:52.891562 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.891566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.891622 (+ 56us) server_negotiation.cc:407] Connection header received
0929 18:43:52.891717 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.891719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.891757 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.891821 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:43:52.892406 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.892905 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.893742 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.893959 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.894039 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.895476 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:43:52.895479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:43:52.895481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:43:52.895506 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:43:52.896943 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.897374 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.897377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.897378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.897461 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:43:52.897736 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:43:52.897738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:43:52.897739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:43:52.897938 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:43:52.898052 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.898246 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.898426 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:43:52.901314 27688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:43:52.904734 27688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:43:52.912873 27810 tablet_service.cc:1467] Processing CreateTablet for tablet 2b885a9c635944e7930606037f03e39d (DEFAULT_TABLE table=test-table [id=2bea322a467e456fb9284dac1abff498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:52.913539 27810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b885a9c635944e7930606037f03e39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:52.916674 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.911403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:41437 (local address 127.0.0.1:46002)
0929 18:43:52.911537 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.911553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.911635 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.912561 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.912565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.912584 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:52.912702 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.912709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.914066 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.914072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.915341 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.915351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.916202 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.916206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.916300 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.916342 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.916431 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.916513 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 18:43:52.916911 27768 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Bootstrap starting.
I20260929 18:43:52.916944 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.911558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.11.118.195:41437)
0929 18:43:52.911984 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.912002 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.912028 (+ 26us) server_negotiation.cc:407] Connection header received
0929 18:43:52.912111 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.912117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.912365 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.912569 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:52.912800 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.913950 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.915473 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.916063 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.916221 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.916366 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.916464 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.916717 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":152,"threads_started":1}
I20260929 18:43:52.917877 27768 tablet_bootstrap.cc:654] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:52.918210 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.911883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:42461 (local address 127.0.0.1:38760)
0929 18:43:52.912127 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.912150 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.912232 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.912951 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.912954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.912971 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:52.913065 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.913071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.914966 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.914969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.916075 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.916089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.917734 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.917741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.917865 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.917895 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.917975 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.918052 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260929 18:43:52.918778 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.912091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.11.118.194:42461)
0929 18:43:52.912479 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.912514 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.912537 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:43:52.912606 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.912611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.912808 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.912968 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:52.913260 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.914834 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.916238 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.917604 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.917724 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.917838 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.918439 (+ 601us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.918614 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":108,"threads_started":1}
I20260929 18:43:52.919518 27768 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: No bootstrap required, opened a new log
I20260929 18:43:52.919610 27768 ts_tablet_manager.cc:1397] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 18:43:52.920094 27768 raft_consensus.cc:348] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.920219 27768 raft_consensus.cc:374] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:52.920265 27768 raft_consensus.cc:729] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed916a48cceb48a9a1efa2367e85416a, State: Initialized, Role: FOLLOWER
I20260929 18:43:52.920403 27768 consensus_queue.cc:260] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.920686 27768 ts_tablet_manager.cc:1428] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 18:43:52.920447 28080 tablet_service.cc:1467] Processing CreateTablet for tablet 2b885a9c635944e7930606037f03e39d (DEFAULT_TABLE table=test-table [id=2bea322a467e456fb9284dac1abff498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:52.921562 27948 tablet_service.cc:1467] Processing CreateTablet for tablet 2b885a9c635944e7930606037f03e39d (DEFAULT_TABLE table=test-table [id=2bea322a467e456fb9284dac1abff498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:43:52.922508 28080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b885a9c635944e7930606037f03e39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:52.922574 27948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b885a9c635944e7930606037f03e39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:43:52.931030 28159 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Bootstrap starting.
I20260929 18:43:52.931360 28160 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Bootstrap starting.
I20260929 18:43:52.933979 28160 tablet_bootstrap.cc:654] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:52.933977 28159 tablet_bootstrap.cc:654] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Neither blocks nor log segments found. Creating new log.
I20260929 18:43:52.934943 28159 log.cc:824] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:52.934947 28160 log.cc:824] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Log is configured to *not* fsync() on all Append() calls
I20260929 18:43:52.936945 28160 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: No bootstrap required, opened a new log
I20260929 18:43:52.936960 28159 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: No bootstrap required, opened a new log
I20260929 18:43:52.937178 28160 ts_tablet_manager.cc:1397] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 18:43:52.937179 28159 ts_tablet_manager.cc:1397] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 18:43:52.940214 28160 raft_consensus.cc:348] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.940215 28159 raft_consensus.cc:348] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.940486 28160 raft_consensus.cc:374] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:52.940481 28159 raft_consensus.cc:374] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:43:52.940574 28159 raft_consensus.cc:729] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, State: Initialized, Role: FOLLOWER
I20260929 18:43:52.940569 28160 raft_consensus.cc:729] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5ae1b80e9d4e5e9b844a135b915b07, State: Initialized, Role: FOLLOWER
I20260929 18:43:52.941066 28160 consensus_queue.cc:260] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.941092 28159 consensus_queue.cc:260] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.941922 28014 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
I20260929 18:43:52.942070 28146 heartbeater.cc:503] Master 127.11.118.254:36281 was elected leader, sending a full tablet report...
I20260929 18:43:52.942135 28159 ts_tablet_manager.cc:1428] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260929 18:43:52.942152 28160 ts_tablet_manager.cc:1428] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260929 18:43:52.946455 27877 tablet.cc:2369] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:43:52.948072 27882 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:43:52.948279 27882 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.949378 27882 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
I20260929 18:43:52.953238 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.949803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:43047 (local address 127.11.118.195:41437)
0929 18:43:52.949914 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.949919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.949933 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:52.949980 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.949984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.950036 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.950125 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:52.950508 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.951058 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952202 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.952610 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952746 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.952793 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.953041 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.953110 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:43:52.953253 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.949539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:41437 (local address 127.11.118.193:43047)
0929 18:43:52.949664 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.949680 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.949779 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.950164 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.950168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.950187 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:52.950383 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.950390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.951166 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.951170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.952079 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.952088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.952744 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.952903 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.952939 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.952999 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.953094 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":75}
I20260929 18:43:52.953253 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.949866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:39603 (local address 127.11.118.194:42461)
0929 18:43:52.949977 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.949980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.950030 (+ 50us) server_negotiation.cc:407] Connection header received
0929 18:43:52.950190 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.950193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.950237 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.950347 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:43:52.950638 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.951125 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952207 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.952603 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952734 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.952786 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.953072 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.953136 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 18:43:52.953279 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.949618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:42461 (local address 127.11.118.193:39603)
0929 18:43:52.949965 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:52.949977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:43:52.950085 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:43:52.950372 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:43:52.950375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:43:52.950398 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:43:52.950533 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.950539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.951239 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.951242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.952083 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:43:52.952092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.952739 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.952743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:43:52.952915 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.952947 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:43:52.953040 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 18:43:52.953110 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":66,"thread_start_us":91,"threads_started":1}
I20260929 18:43:52.953845 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b885a9c635944e7930606037f03e39d" candidate_uuid: "ed916a48cceb48a9a1efa2367e85416a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" is_pre_election: true
I20260929 18:43:52.953868 27968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b885a9c635944e7930606037f03e39d" candidate_uuid: "ed916a48cceb48a9a1efa2367e85416a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" is_pre_election: true
I20260929 18:43:52.954133 27968 raft_consensus.cc:2454] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed916a48cceb48a9a1efa2367e85416a in term 0.
I20260929 18:43:52.954138 28100 raft_consensus.cc:2454] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed916a48cceb48a9a1efa2367e85416a in term 0.
I20260929 18:43:52.954674 27764 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07, ed916a48cceb48a9a1efa2367e85416a; no voters:
I20260929 18:43:52.954950 27882 raft_consensus.cc:2781] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:43:52.955022 27882 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:43:52.955075 27882 raft_consensus.cc:3037] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:52.956096 27882 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.956502 27882 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 1 election: Requested vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
I20260929 18:43:52.956808 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b885a9c635944e7930606037f03e39d" candidate_uuid: "ed916a48cceb48a9a1efa2367e85416a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
I20260929 18:43:52.956925 27968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b885a9c635944e7930606037f03e39d" candidate_uuid: "ed916a48cceb48a9a1efa2367e85416a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
I20260929 18:43:52.956936 28100 raft_consensus.cc:3037] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:52.957052 27968 raft_consensus.cc:3037] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:43:52.958047 28100 raft_consensus.cc:2454] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed916a48cceb48a9a1efa2367e85416a in term 1.
I20260929 18:43:52.958168 27968 raft_consensus.cc:2454] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed916a48cceb48a9a1efa2367e85416a in term 1.
I20260929 18:43:52.958447 27763 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, ed916a48cceb48a9a1efa2367e85416a; no voters:
I20260929 18:43:52.958640 27882 raft_consensus.cc:2781] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:43:52.958889 27882 raft_consensus.cc:686] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 LEADER]: Becoming Leader. State: Replica: ed916a48cceb48a9a1efa2367e85416a, State: Running, Role: LEADER
I20260929 18:43:52.959157 27882 consensus_queue.cc:237] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:43:52.961974 27688 catalog_manager.cc:5495] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a reported cstate change: term changed from 0 to 1, leader changed from <none> to ed916a48cceb48a9a1efa2367e85416a (127.11.118.193). New cstate: current_term: 1 leader_uuid: "ed916a48cceb48a9a1efa2367e85416a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } health_report { overall_health: HEALTHY } } }
I20260929 18:43:52.975569 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:52.972316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.11.118.193:45327)
0929 18:43:52.972440 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 18:43:52.972443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:43:52.972457 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:43:52.972560 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:43:52.972562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:43:52.972619 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:43:52.972708 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:43:52.972912 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.973479 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.974252 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:43:52.974497 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:43:52.974572 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:43:52.974786 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:43:52.975247 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:43:52.975333 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:43:52.975404 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 18:43:52.975454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 18:43:52.979343 27968 raft_consensus.cc:1261] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Refusing update from remote peer ed916a48cceb48a9a1efa2367e85416a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:52.979344 28100 raft_consensus.cc:1261] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Refusing update from remote peer ed916a48cceb48a9a1efa2367e85416a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:43:52.980055 27882 consensus_queue.cc:1035] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:52.980257 28166 consensus_queue.cc:1035] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:43:52.996191 27688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:52.996372 27688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:43:52.998665 27688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Sending DeleteTablet for 3 replicas of tablet 2b885a9c635944e7930606037f03e39d
I20260929 18:43:52.999418 28080 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b885a9c635944e7930606037f03e39d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:52 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46002
I20260929 18:43:52.999472 27948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b885a9c635944e7930606037f03e39d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:52 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:38760
I20260929 18:43:52.999531 27810 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b885a9c635944e7930606037f03e39d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:43:52 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57324
I20260929 18:43:52.999859 28179 tablet_replica.cc:330] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: stopping tablet replica
I20260929 18:43:52.999918 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27726
I20260929 18:43:52.999925 28180 tablet_replica.cc:330] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: stopping tablet replica
I20260929 18:43:53.000809 28181 tablet_replica.cc:330] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: stopping tablet replica
I20260929 18:43:53.000873 28180 raft_consensus.cc:2229] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:53.001082 28181 raft_consensus.cc:2229] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:43:53.001195 28180 raft_consensus.cc:2258] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:53.001348 28181 raft_consensus.cc:2258] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:43:53.003089 28181 ts_tablet_manager.cc:1905] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:53.003433 28180 ts_tablet_manager.cc:1905] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:43:53.005290 28181 ts_tablet_manager.cc:1918] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:53.005406 28181 log.cc:1196] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/2b885a9c635944e7930606037f03e39d
I20260929 18:43:53.005724 28181 ts_tablet_manager.cc:1939] T 2b885a9c635944e7930606037f03e39d P 6e5ae1b80e9d4e5e9b844a135b915b07: Deleting consensus metadata
I20260929 18:43:53.006904 28180 ts_tablet_manager.cc:1918] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:43:53.006925 27674 catalog_manager.cc:4841] TS 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): tablet 2b885a9c635944e7930606037f03e39d (table test-table [id=2bea322a467e456fb9284dac1abff498]) successfully deleted
I20260929 18:43:53.007007 28180 log.cc:1196] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/2b885a9c635944e7930606037f03e39d
I20260929 18:43:53.007364 28180 ts_tablet_manager.cc:1939] T 2b885a9c635944e7930606037f03e39d P 2cd2fdf0f1174d9c80ecb0f4cfcb8feb: Deleting consensus metadata
W20260929 18:43:53.008023 27674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:45327 (error 108)
I20260929 18:43:53.008579 27673 catalog_manager.cc:4841] TS 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): tablet 2b885a9c635944e7930606037f03e39d (table test-table [id=2bea322a467e456fb9284dac1abff498]) successfully deleted
I20260929 18:43:53.009025 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:43:53.008763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:45327 (local address 127.0.0.1:57336)
0929 18:43:53.008896 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 18:43:53.008954 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 18:43:53.009191 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27878
W20260929 18:43:53.009231 27674 catalog_manager.cc:4584] TS ed916a48cceb48a9a1efa2367e85416a (127.11.118.193:45327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b885a9c635944e7930606037f03e39d: Network error: Client connection negotiation failed: client connection to 127.11.118.193:45327: connect: Connection refused (error 111)
I20260929 18:43:53.015097 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28017
I20260929 18:43:53.020740 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 27655
I20260929 18:44:13.028564 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--webserver_interface=127.11.118.254
--webserver_port=36857
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:13.142794 28183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:13.143074 28183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:13.143115 28183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:13.147271 28183 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 18:44:13.147341 28183 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 18:44:13.147358 28183 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 18:44:13.147372 28183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:13.147392 28183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:13.147440 28183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:13.147475 28183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:13.147496 28183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:13.153051 28183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:36281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:13.154213 28183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:13.155287 28183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:13.164192 28188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.164301 28183 server_base.cc:1034] running on GCE node
W20260929 18:44:13.164230 28189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.164239 28191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.164959 28183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:13.166085 28183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:13.167256 28183 hybrid_clock.cc:648] HybridClock initialized: now 1790707453167248 us; error 32 us; skew 500 ppm
Sep 29 18:44:13 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:13.170063 28183 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:13.171113 28183 webserver.cc:460] Webserver started at http://127.11.118.254:36857/ using document root <none> and password file <none>
I20260929 18:44:13.171660 28183 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:13.171721 28183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:13.175683 28183 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260929 18:44:13.177942 28198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.179556 28183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:44:13.179718 28183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "e5d2a34f41b746139b4141fe33864444"
format_stamp: "Formatted at 2026-09-29 18:43:30 on dist-test-slave-cgv7"
server_key: "7bbe4f6ab766ff6a471aa3ba522d3916"
server_key_iv: "b1b9e1b4b1c1d878769440cdbdfe3bde"
server_key_version: "encryptionkey@0"
I20260929 18:44:13.180341 28183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:13.207312 28183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:13.210491 28183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:13.210696 28183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:13.217914 28183 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:36281
I20260929 18:44:13.218261 28250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:36281 every 8 connection(s)
I20260929 18:44:13.219080 28183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:13.224433 28251 sys_catalog.cc:263] Verifying existing consensus state
I20260929 18:44:13.225770 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28183
I20260929 18:44:13.226224 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:45327
--local_ip_for_outbound_sockets=127.11.118.193
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=37359
--webserver_interface=127.11.118.193
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:44:13.226418 28251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap starting.
I20260929 18:44:13.244135 28251 log.cc:824] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:13.255625 28251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:44:13.256064 28251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Bootstrap complete.
I20260929 18:44:13.260185 28251 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:44:13.260619 28251 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Initialized, Role: FOLLOWER
I20260929 18:44:13.261153 28251 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:44:13.261304 28251 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:13.261395 28251 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:13.261520 28251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 2 FOLLOWER]: Advancing to term 3
I20260929 18:44:13.263625 28251 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:44:13.264014 28251 leader_election.cc:302] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5d2a34f41b746139b4141fe33864444; no voters:
I20260929 18:44:13.264348 28251 leader_election.cc:288] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 18:44:13.264497 28255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 18:44:13.264760 28255 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [term 3 LEADER]: Becoming Leader. State: Replica: e5d2a34f41b746139b4141fe33864444, State: Running, Role: LEADER
I20260929 18:44:13.265103 28255 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } }
I20260929 18:44:13.265578 28251 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:13.266137 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5d2a34f41b746139b4141fe33864444. Latest consensus state: current_term: 3 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:44:13.266253 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:13.266881 28256 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e5d2a34f41b746139b4141fe33864444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d2a34f41b746139b4141fe33864444" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36281 } } }
I20260929 18:44:13.266984 28256 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:13.267759 28260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:13.273936 28260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:13.274631 28260 catalog_manager.cc:1259] Loaded cluster ID: d60eeaf5c69746f8aa1a66d754821de9
I20260929 18:44:13.274693 28260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:13.277104 28260 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:13.277840 28260 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Loaded TSK: 0
I20260929 18:44:13.277927 28260 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5d2a34f41b746139b4141fe33864444: Loaded TSK: 1
F20260929 18:44:13.278700 28260 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4fde3bf9ad google::LogMessage::Fail() at ??:0
@ 0x7f4fde3c288a google::LogMessage::SendToLog() at ??:0
@ 0x7f4fde3bf4de google::LogMessage::Flush() at ??:0
@ 0x7f4fde3c0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4fe108318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4fe107bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4fe8d247bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4fe8cfe99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4fe8cff178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f4fe8d40df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4fe8d5f82b std::function<>::operator()() at ??:0
@ 0x7f4fe8cfeb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4fe8cfff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4fe8cfc579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4fe8d40640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4fe06113d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4fe0611c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4fe06133f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4fe0602512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4fe1c466db start_thread at ??:0
@ 0x7f4fdcebd61f clone at ??:0
W20260929 18:44:13.385349 28253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:13.385627 28253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:13.385676 28253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:13.389920 28253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:13.389991 28253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:13.390053 28253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:13.395844 28253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:45327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=37359
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:13.397037 28253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:13.398093 28253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:13.409144 28299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.409099 28298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.409457 28301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.409502 28253 server_base.cc:1034] running on GCE node
I20260929 18:44:13.410054 28253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:13.410810 28253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:13.412007 28253 hybrid_clock.cc:648] HybridClock initialized: now 1790707453411985 us; error 39 us; skew 500 ppm
Sep 29 18:44:13 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:13.414953 28253 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:44:13.415908 28253 webserver.cc:460] Webserver started at http://127.11.118.193:37359/ using document root <none> and password file <none>
I20260929 18:44:13.416504 28253 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:13.416566 28253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:13.420591 28253 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:44:13.422967 28308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.424278 28253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:13.424427 28253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "ed916a48cceb48a9a1efa2367e85416a"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "4b8ac661c0552dc0e89bcd514c8693a4"
server_key_iv: "65e4e11257f6f8163fab4fc7318dd25a"
server_key_version: "encryptionkey@0"
I20260929 18:44:13.424983 28253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:13.447489 28253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:13.450899 28253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:13.451114 28253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:13.451730 28253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:13.452875 28315 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 18:44:13.458259 28253 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 18:44:13.458400 28253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260929 18:44:13.458492 28253 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 18:44:13.460132 28253 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 18:44:13.460207 28253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 18:44:13.460500 28315 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Bootstrap starting.
I20260929 18:44:13.471376 28253 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:45327
I20260929 18:44:13.471616 28422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:45327 every 8 connection(s)
I20260929 18:44:13.472431 28253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:13.473680 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28253
I20260929 18:44:13.474179 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:42461
--local_ip_for_outbound_sockets=127.11.118.194
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=46123
--webserver_interface=127.11.118.194
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:44:13.482656 28315 log.cc:824] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:13.492259 28315 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 18:44:13.492712 28315 tablet_bootstrap.cc:492] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Bootstrap complete.
I20260929 18:44:13.494163 28315 ts_tablet_manager.cc:1397] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.008s
I20260929 18:44:13.497547 28315 raft_consensus.cc:348] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:13.498172 28315 raft_consensus.cc:729] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed916a48cceb48a9a1efa2367e85416a, State: Initialized, Role: FOLLOWER
I20260929 18:44:13.498713 28315 consensus_queue.cc:260] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:13.499787 28315 ts_tablet_manager.cc:1428] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
W20260929 18:44:13.510679 28425 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:44:13.474766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:43669)
0929 18:44:13.476098 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.476159 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:13.477387 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:13.509787 (+ 32400us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":176,"threads_started":1}
W20260929 18:44:13.511255 28423 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:36281 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:36281: Client connection negotiation failed: client connection to 127.11.118.254:36281: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 18:44:13.511996 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.511754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:42983)
0929 18:44:13.511878 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.511942 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:44:13.512696 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.512506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:54937)
0929 18:44:13.512612 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.512652 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 18:44:13.512952 28423 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 18:44:13.597363 28426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:13.597680 28426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:13.597728 28426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:13.602178 28426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:13.602257 28426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:13.602383 28426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:13.608042 28426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:42461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=46123
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:13.609257 28426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:13.610436 28426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:13.620528 28434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.620517 28436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.620571 28426 server_base.cc:1034] running on GCE node
W20260929 18:44:13.620517 28433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.621256 28426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:13.622014 28426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:13.623236 28426 hybrid_clock.cc:648] HybridClock initialized: now 1790707453623209 us; error 48 us; skew 500 ppm
Sep 29 18:44:13 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:13.626176 28426 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:44:13.627228 28426 webserver.cc:460] Webserver started at http://127.11.118.194:46123/ using document root <none> and password file <none>
I20260929 18:44:13.627815 28426 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:13.627892 28426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:13.632019 28426 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:44:13.634223 28443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.635552 28426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:13.635690 28426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "6e5ae1b80e9d4e5e9b844a135b915b07"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "b4ff265e2f365ad707a448a71121ffb5"
server_key_iv: "394c553de2268f278ca5490a844e190b"
server_key_version: "encryptionkey@0"
I20260929 18:44:13.636268 28426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:13.665381 28426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:13.669168 28426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:13.669394 28426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:13.670226 28426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:13.671485 28426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:13.671562 28426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.671638 28426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:13.671676 28426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.682516 28426 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:42461
I20260929 18:44:13.682534 28556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:42461 every 8 connection(s)
I20260929 18:44:13.683665 28426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:13.687642 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.685802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:44955)
0929 18:44:13.686774 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.686901 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":606,"threads_started":1}
W20260929 18:44:13.688092 28557 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:36281 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:36281: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
I20260929 18:44:13.688710 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.688505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:49213)
0929 18:44:13.688612 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.688660 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 18:44:13.689327 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.689134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:50373)
0929 18:44:13.689240 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.689281 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 18:44:13.689507 28557 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 18:44:13.693416 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28426
I20260929 18:44:13.693845 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:41437
--local_ip_for_outbound_sockets=127.11.118.195
--tserver_master_addrs=127.11.118.254:36281
--webserver_port=36513
--webserver_interface=127.11.118.195
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:13.806172 28560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:13.806528 28560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:13.806614 28560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:13.810702 28560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:13.810786 28560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:13.810879 28560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:13.816340 28560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:41437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=36513
--tserver_master_addrs=127.11.118.254:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:13.817438 28560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:13.818440 28560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:13.828266 28565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.828266 28568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:13.828280 28566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:13.828547 28560 server_base.cc:1034] running on GCE node
I20260929 18:44:13.829037 28560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:13.829696 28560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:13.830889 28560 hybrid_clock.cc:648] HybridClock initialized: now 1790707453830863 us; error 44 us; skew 500 ppm
Sep 29 18:44:13 dist-test-slave-cgv7 krb5kdc[27108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:13.833755 28560 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:44:13.834692 28560 webserver.cc:460] Webserver started at http://127.11.118.195:36513/ using document root <none> and password file <none>
I20260929 18:44:13.835261 28560 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:13.835356 28560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:13.839263 28560 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:44:13.841593 28575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.842916 28560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:44:13.843086 28560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb"
format_stamp: "Formatted at 2026-09-29 18:43:31 on dist-test-slave-cgv7"
server_key: "2a51bf9019293d98e9c27886017ae7d8"
server_key_iv: "9d56d2a3278dd66f795e96193478cfcc"
server_key_version: "encryptionkey@0"
I20260929 18:44:13.843648 28560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:13.868219 28560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:13.871647 28560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:13.871868 28560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:13.872653 28560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:13.873588 28560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:13.873669 28560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.873744 28560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:13.873791 28560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:13.882964 28560 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:41437
I20260929 18:44:13.882983 28688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:41437 every 8 connection(s)
I20260929 18:44:13.883980 28560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:13.887777 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.886091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:51191)
0929 18:44:13.886634 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.887056 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":186,"threads_started":1}
W20260929 18:44:13.888244 28689 heartbeater.cc:644] Failed to heartbeat to 127.11.118.254:36281 (0 consecutive failures): Network error: Failed to ping master at 127.11.118.254:36281: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
I20260929 18:44:13.888846 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.888648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:53541)
0929 18:44:13.888749 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.888796 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 18:44:13.889417 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:13.889245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:51279)
0929 18:44:13.889331 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:13.889370 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 18:44:13.889599 28689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 18:44:13.890846 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28560
I20260929 18:44:14.514107 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.513635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:38467)
0929 18:44:14.513938 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:14.514002 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":103,"threads_started":1}
I20260929 18:44:14.638676 28694 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:14.638918 28694 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:14.640197 28694 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
Sep 29 18:44:14 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.194@KRBTEST.COM
Sep 29 18:44:14 dist-test-slave-cgv7 krb5kdc[27108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707453, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.195@KRBTEST.COM
I20260929 18:44:14.657310 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.640546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:41437 (local address 127.11.118.193:50567)
0929 18:44:14.640801 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:14.640922 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:14.641039 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:14.643050 (+ 2011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:14.643057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:14.643723 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:14.644649 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:14.644672 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.646719 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.646722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:14.647841 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:14.647856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.648993 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.648997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:14.649097 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:14.649820 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:14.649833 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:14.651919 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:14.654492 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:14.654501 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:14.654516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:14.654910 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:14.655347 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:14.655354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:14.655532 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:14.656274 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:14.656281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:14.656551 (+ 270us) client_negotiation.cc:769] Sending connection context
0929 18:44:14.656743 (+ 192us) client_negotiation.cc:240] Negotiation successful
0929 18:44:14.657101 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":282,"spinlock_wait_cycles":21760}
I20260929 18:44:14.657354 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.640546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:42461 (local address 127.11.118.193:41539)
0929 18:44:14.640941 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:14.640956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:14.641068 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:14.643050 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:14.643057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:14.643723 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:14.644649 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:14.644672 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.646707 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.646710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:14.647841 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:14.647855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.648821 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.648825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:14.649028 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:14.649753 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:14.649770 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:14.651563 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:14.654844 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:14.654846 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:14.654847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:14.655141 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:14.655544 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:14.655548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:14.655653 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:14.656315 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:14.656318 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:14.656560 (+ 242us) client_negotiation.cc:769] Sending connection context
0929 18:44:14.656742 (+ 182us) client_negotiation.cc:240] Negotiation successful
0929 18:44:14.657191 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":378,"spinlock_wait_cycles":43264,"thread_start_us":81,"threads_started":1}
I20260929 18:44:14.657371 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.640694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:50567 (local address 127.11.118.195:41437)
0929 18:44:14.641136 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:14.641153 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:14.641200 (+ 47us) server_negotiation.cc:407] Connection header received
0929 18:44:14.642067 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:14.642075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:14.642583 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:14.642931 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:14.644889 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.646543 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.648239 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.648833 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.649009 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:14.652180 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:14.652186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:14.652203 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:14.652243 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:14.654343 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655032 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:14.655040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:14.655047 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:14.655213 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655636 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:14.655640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:14.655642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:14.656138 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:14.656262 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:14.656859 (+ 597us) server_negotiation.cc:299] Negotiation successful
0929 18:44:14.657178 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":95,"threads_started":1}
I20260929 18:44:14.657341 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.640694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:41539 (local address 127.11.118.194:42461)
0929 18:44:14.641157 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:14.641168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:14.641200 (+ 32us) server_negotiation.cc:407] Connection header received
0929 18:44:14.642067 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:14.642075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:14.642583 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:14.642931 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:14.644919 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.646513 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.648089 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:14.648678 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:14.648824 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:14.651927 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:14.651935 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:14.651957 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:14.652003 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:14.654718 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655253 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:14.655276 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:14.655284 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:14.655443 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:14.655745 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:14.655747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:14.655748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:14.656164 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:14.656305 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:14.656920 (+ 615us) server_negotiation.cc:299] Negotiation successful
0929 18:44:14.657182 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":111,"threads_started":1}
W20260929 18:44:14.660061 28310 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
W20260929 18:44:14.660457 28311 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
I20260929 18:44:14.660533 28311 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed916a48cceb48a9a1efa2367e85416a; no voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07
I20260929 18:44:14.660833 28694 raft_consensus.cc:2726] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:44:14.690747 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.690225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:53015)
0929 18:44:14.690545 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:14.690620 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
I20260929 18:44:14.890673 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:14.890190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:50747)
0929 18:44:14.890522 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:14.890580 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":179,"threads_started":1}
I20260929 18:44:15.515460 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:15.515016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:38547)
0929 18:44:15.515303 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:15.515359 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
I20260929 18:44:15.692173 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:15.691655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:52247)
0929 18:44:15.691991 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:15.692060 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":125,"threads_started":1}
I20260929 18:44:15.891911 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:15.891453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:50427)
0929 18:44:15.891746 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:15.891803 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
I20260929 18:44:16.516804 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:16.516337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:59169)
0929 18:44:16.516635 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:16.516708 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260929 18:44:16.693459 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:16.693024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:41815)
0929 18:44:16.693291 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:16.693355 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260929 18:44:16.855602 28705 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:16.855737 28705 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:16.856204 28705 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
W20260929 18:44:16.857103 28310 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
W20260929 18:44:16.857263 28311 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
I20260929 18:44:16.857340 28311 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed916a48cceb48a9a1efa2367e85416a; no voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07
I20260929 18:44:16.857478 28705 raft_consensus.cc:2726] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:44:16.893222 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:16.892778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:39317)
0929 18:44:16.893061 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:16.893127 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
I20260929 18:44:17.518050 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:17.517607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:58823)
0929 18:44:17.517868 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:17.517927 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260929 18:44:17.694684 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:17.694177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:60403)
0929 18:44:17.694523 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:17.694583 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":141,"threads_started":1}
I20260929 18:44:17.894503 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:17.894067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:49453)
0929 18:44:17.894354 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:17.894412 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":88,"threads_started":1}
I20260929 18:44:18.519512 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:18.519050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:59611)
0929 18:44:18.519354 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:18.519417 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":74,"threads_started":1}
I20260929 18:44:18.696203 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:18.695695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:44303)
0929 18:44:18.695996 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:18.696095 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
I20260929 18:44:18.895764 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:18.895342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:49553)
0929 18:44:18.895608 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:18.895670 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260929 18:44:19.378432 28713 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:19.378605 28713 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:19.379104 28713 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
W20260929 18:44:19.379914 28311 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
W20260929 18:44:19.380038 28310 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
I20260929 18:44:19.380138 28310 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed916a48cceb48a9a1efa2367e85416a; no voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07
I20260929 18:44:19.380335 28713 raft_consensus.cc:2726] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:44:19.520804 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:19.520376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:53613)
0929 18:44:19.520650 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:19.520708 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260929 18:44:19.697618 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:19.697020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:59679)
0929 18:44:19.697410 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:19.697499 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":79,"threads_started":1}
I20260929 18:44:19.897089 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:19.896585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:56465)
0929 18:44:19.896925 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:19.896987 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":81,"threads_started":1}
I20260929 18:44:20.522326 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:20.521816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:34859)
0929 18:44:20.522107 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:20.522174 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260929 18:44:20.699159 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:20.698663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:57993)
0929 18:44:20.698995 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:20.699062 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":81,"threads_started":1}
I20260929 18:44:20.898734 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:20.898128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:39549)
0929 18:44:20.898551 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:20.898619 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":152,"threads_started":1}
I20260929 18:44:21.523916 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:21.523486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:36521)
0929 18:44:21.523746 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:21.523816 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260929 18:44:21.700363 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:21.699931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:56237)
0929 18:44:21.700196 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:21.700253 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":73,"threads_started":1}
I20260929 18:44:21.899914 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:21.899523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:58665)
0929 18:44:21.899762 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:21.899817 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260929 18:44:22.525273 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:22.524773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:36339)
0929 18:44:22.525109 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:22.525170 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":85,"threads_started":1}
I20260929 18:44:22.701807 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:22.701287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:42141)
0929 18:44:22.701618 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:22.701705 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260929 18:44:22.901487 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:22.900955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:37455)
0929 18:44:22.901315 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:22.901379 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":121,"threads_started":1}
I20260929 18:44:23.526857 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:23.526367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:50143)
0929 18:44:23.526671 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:23.526749 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":78,"threads_started":1}
I20260929 18:44:23.703227 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:23.702742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:57247)
0929 18:44:23.703064 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:23.703120 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":92,"threads_started":1}
I20260929 18:44:23.902844 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:23.902429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:38653)
0929 18:44:23.902686 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:23.902747 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260929 18:44:24.237051 28729 raft_consensus.cc:482] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:24.237174 28729 raft_consensus.cc:504] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd2fdf0f1174d9c80ecb0f4cfcb8feb" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 41437 } } peers { permanent_uuid: "6e5ae1b80e9d4e5e9b844a135b915b07" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42461 } } peers { permanent_uuid: "ed916a48cceb48a9a1efa2367e85416a" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 45327 } }
I20260929 18:44:24.237591 28729 leader_election.cc:288] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437), 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461)
W20260929 18:44:24.238402 28311 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e5ae1b80e9d4e5e9b844a135b915b07 (127.11.118.194:42461): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
W20260929 18:44:24.238492 28310 leader_election.cc:341] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cd2fdf0f1174d9c80ecb0f4cfcb8feb (127.11.118.195:41437): Not found: Tablet not found: 2b885a9c635944e7930606037f03e39d
I20260929 18:44:24.238603 28310 leader_election.cc:302] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed916a48cceb48a9a1efa2367e85416a; no voters: 2cd2fdf0f1174d9c80ecb0f4cfcb8feb, 6e5ae1b80e9d4e5e9b844a135b915b07
I20260929 18:44:24.238763 28729 raft_consensus.cc:2726] T 2b885a9c635944e7930606037f03e39d P ed916a48cceb48a9a1efa2367e85416a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 18:44:24.528267 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:24.527727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:48177)
0929 18:44:24.528080 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:24.528150 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260929 18:44:24.704624 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:24.704112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:49697)
0929 18:44:24.704410 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:24.704484 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":72,"threads_started":1}
I20260929 18:44:24.904166 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:24.903670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:45761)
0929 18:44:24.904001 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:24.904063 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":128,"threads_started":1}
I20260929 18:44:25.529543 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:25.529107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:59855)
0929 18:44:25.529374 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:25.529439 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260929 18:44:25.705905 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:25.705457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:38503)
0929 18:44:25.705745 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:25.705810 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260929 18:44:25.905551 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:25.905087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:39803)
0929 18:44:25.905356 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:25.905410 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260929 18:44:26.530894 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:26.530442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:44913)
0929 18:44:26.530700 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:26.530767 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260929 18:44:26.707279 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:26.706806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:43059)
0929 18:44:26.707105 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:26.707172 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260929 18:44:26.906945 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:26.906528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:42165)
0929 18:44:26.906776 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:26.906837 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260929 18:44:27.532186 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:27.531750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:49979)
0929 18:44:27.532021 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:27.532086 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260929 18:44:27.708671 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:27.708210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:33767)
0929 18:44:27.708503 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:27.708572 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":82,"threads_started":1}
I20260929 18:44:27.908354 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:27.907895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.195:48965)
0929 18:44:27.908178 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:27.908245 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260929 18:44:28.533665 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:28.533250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.193:44679)
0929 18:44:28.533519 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:28.533571 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260929 18:44:28.710045 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:28.709648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36281 (local address 127.11.118.194:42729)
0929 18:44:28.709898 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:28.709951 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.254:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260929 18:44:28.895681 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28253
I20260929 18:44:28.901551 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28426
I20260929 18:44:28.907011 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28560
2026-09-29T18:44:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59510 ms)
[----------] 30 tests from SecurityITest (245743 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:28 dist-test-slave-cgv7 krb5kdc[28746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:28 dist-test-slave-cgv7 krb5kdc[28746](info): set up 2 sockets
Sep 29 18:44:28 dist-test-slave-cgv7 krb5kdc[28746](info): commencing operation
krb5kdc: starting...
W20260929 18:44:29.978607 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:44:30.107556 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36617
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:30.217665 28759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:30.217935 28759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:30.217976 28759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:30.221943 28759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:30.222014 28759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:30.222033 28759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:30.222050 28759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:30.222065 28759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:30.227370 28759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:30.228426 28759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:30.229517 28759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:30.238346 28767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.238457 28765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.238395 28764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:30.238406 28759 server_base.cc:1034] running on GCE node
I20260929 18:44:30.239293 28759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:30.240458 28759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:30.241652 28759 hybrid_clock.cc:648] HybridClock initialized: now 1790707470241636 us; error 31 us; skew 500 ppm
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:30.244441 28759 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:30.245380 28759 webserver.cc:460] Webserver started at http://127.11.118.254:43117/ using document root <none> and password file <none>
I20260929 18:44:30.245970 28759 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:30.246062 28759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:30.246333 28759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:30.248400 28759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "1b915267b0824c148562cc6ebfa9765c"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "b4742fccad1d24440fba0d088156fd58"
server_key_iv: "9795214a80ebbae2cc026cce17c12a96"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.248970 28759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "1b915267b0824c148562cc6ebfa9765c"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "b4742fccad1d24440fba0d088156fd58"
server_key_iv: "9795214a80ebbae2cc026cce17c12a96"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.252733 28759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260929 18:44:30.255353 28774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.256695 28759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 18:44:30.257025 28759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "1b915267b0824c148562cc6ebfa9765c"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "b4742fccad1d24440fba0d088156fd58"
server_key_iv: "9795214a80ebbae2cc026cce17c12a96"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.257216 28759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:30.292055 28759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:30.295518 28759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:30.295763 28759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:30.303819 28759 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:36617
I20260929 18:44:30.303825 28826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:36617 every 8 connection(s)
I20260929 18:44:30.304865 28759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:30.307813 28827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:30.313767 28827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: Bootstrap starting.
I20260929 18:44:30.314507 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28759
I20260929 18:44:30.314646 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:30.314952 11739 external_mini_cluster.cc:1428] Setting key 9e5e05e687370e6e25902722ab7cd772
I20260929 18:44:30.316933 28827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:30.317710 28827 log.cc:824] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:30.320099 28827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: No bootstrap required, opened a new log
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:30.325017 28827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } }
I20260929 18:44:30.325304 28827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:30.325392 28827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b915267b0824c148562cc6ebfa9765c, State: Initialized, Role: FOLLOWER
I20260929 18:44:30.325951 28827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } }
I20260929 18:44:30.326128 28827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:30.326197 28827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:30.326344 28827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:30.327315 28827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } }
I20260929 18:44:30.327679 28827 leader_election.cc:302] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b915267b0824c148562cc6ebfa9765c; no voters:
I20260929 18:44:30.328094 28827 leader_election.cc:288] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:30.328176 28832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:30.328436 28832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [term 1 LEADER]: Becoming Leader. State: Replica: 1b915267b0824c148562cc6ebfa9765c, State: Running, Role: LEADER
I20260929 18:44:30.328837 28832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } }
I20260929 18:44:30.329485 28827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:30.330816 28833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b915267b0824c148562cc6ebfa9765c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } } }
I20260929 18:44:30.331043 28833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:30.331254 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:30.316401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44272 (local address 127.11.118.254:36617)
0929 18:44:30.316889 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:30.316906 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:30.316944 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:44:30.317683 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:30.317702 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:30.318136 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:30.318595 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:30.319446 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.320514 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.321700 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.322202 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.322409 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:30.325604 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:30.325611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:30.325629 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:30.325668 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:30.327689 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.328283 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.328290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.328298 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.328477 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.328811 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.328814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.328815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.329367 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:30.329565 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:30.329892 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 18:44:30.330216 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":161,"threads_started":1}
I20260929 18:44:30.331532 28842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:30.332576 28834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b915267b0824c148562cc6ebfa9765c. Latest consensus state: current_term: 1 leader_uuid: "1b915267b0824c148562cc6ebfa9765c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b915267b0824c148562cc6ebfa9765c" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 36617 } } }
I20260929 18:44:30.332702 28834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:30.334980 28842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:30.340867 28842 catalog_manager.cc:1347] Generated new cluster ID: 421d96d9471c4581a002fd024b6317e3
I20260929 18:44:30.340945 28842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:30.361202 28842 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:30.362499 28842 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:30.373354 28842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: Generated new TSK 0
I20260929 18:44:30.374126 28842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:44:30.414338 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:30.530082 28853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:30.530459 28853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:30.530536 28853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:30.534730 28853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:30.534801 28853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:30.534869 28853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:30.540371 28853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:30.541496 28853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:30.542549 28853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:44:30.552789 28853 server_base.cc:1034] running on GCE node
W20260929 18:44:30.552778 28859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.552765 28858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.552814 28861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:30.553506 28853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:30.554150 28853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:30.555384 28853 hybrid_clock.cc:648] HybridClock initialized: now 1790707470555348 us; error 53 us; skew 500 ppm
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:30.558818 28853 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:44:30.559844 28853 webserver.cc:460] Webserver started at http://127.11.118.193:33605/ using document root <none> and password file <none>
I20260929 18:44:30.560541 28853 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:30.560633 28853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:30.560859 28853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:30.563288 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "6459b38c542f47508fb254c2ef60a3f9"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "3bdfe973e70886ccdc30036ec3d9094d"
server_key_iv: "f7f00e4a053041c4da574ffb9badb33a"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.563858 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "6459b38c542f47508fb254c2ef60a3f9"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "3bdfe973e70886ccdc30036ec3d9094d"
server_key_iv: "f7f00e4a053041c4da574ffb9badb33a"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.567970 28853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:44:30.570741 28868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.572093 28853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:30.572273 28853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "6459b38c542f47508fb254c2ef60a3f9"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "3bdfe973e70886ccdc30036ec3d9094d"
server_key_iv: "f7f00e4a053041c4da574ffb9badb33a"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.572405 28853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:30.592602 28853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:30.596010 28853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:30.596239 28853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:30.596899 28853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:30.597817 28853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:30.597892 28853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.597970 28853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:30.598011 28853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.607191 28853 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:41947
I20260929 18:44:30.607208 28981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:41947 every 8 connection(s)
I20260929 18:44:30.608112 28853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:30.611229 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28853
I20260929 18:44:30.611371 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:30.611699 11739 external_mini_cluster.cc:1428] Setting key 11f5c359cd22ace6f61a2944e9f32367
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:44:30.623086 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:30.609946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:42551 (local address 127.11.118.254:36617)
0929 18:44:30.610114 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:30.610117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:30.611030 (+ 913us) server_negotiation.cc:407] Connection header received
0929 18:44:30.612579 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:30.612583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:30.612641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:30.612753 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:30.614083 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.614694 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.615878 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.616122 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.616241 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:30.619044 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:30.619047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:30.619049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:30.619075 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:30.620651 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.621376 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.621379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.621383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.621467 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.621841 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.621844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.621846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.622039 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:30.622150 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:30.622782 (+ 632us) server_negotiation.cc:299] Negotiation successful
0929 18:44:30.622919 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:44:30.624029 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:30.610368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36617 (local address 127.11.118.193:42551)
0929 18:44:30.610852 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:30.610907 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:30.612365 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:30.612935 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:30.612942 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:30.613426 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:30.613931 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:30.613944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.614829 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.614832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:30.615745 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:30.615754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.616287 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.616299 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:30.616473 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:30.617130 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:30.617148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:30.618840 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:30.620881 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:30.620889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:30.620907 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:30.621254 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:30.621564 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:30.621567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:30.621569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:30.621733 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:30.622146 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:30.622154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:30.622521 (+ 367us) client_negotiation.cc:769] Sending connection context
0929 18:44:30.622735 (+ 214us) client_negotiation.cc:240] Negotiation successful
0929 18:44:30.623095 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":111,"threads_started":1}
I20260929 18:44:30.625499 28982 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36617
I20260929 18:44:30.625794 28982 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:30.626437 28982 heartbeater.cc:511] Master 127.11.118.254:36617 requested a full tablet report, sending...
I20260929 18:44:30.628026 28790 ts_manager.cc:194] Registered new tserver with Master: 6459b38c542f47508fb254c2ef60a3f9 (127.11.118.193:41947)
I20260929 18:44:30.629303 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:42551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:44:30.641988 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:30.753154 28987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:30.753515 28987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:30.753598 28987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:30.757893 28987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:30.757992 28987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:30.758085 28987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:30.763710 28987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:30.764858 28987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:30.765923 28987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:30.776154 28992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.776176 28993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.776175 28995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:30.776428 28987 server_base.cc:1034] running on GCE node
I20260929 18:44:30.776940 28987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:30.777643 28987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:30.778865 28987 hybrid_clock.cc:648] HybridClock initialized: now 1790707470778812 us; error 28 us; skew 500 ppm
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:30.781693 28987 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:44:30.782712 28987 webserver.cc:460] Webserver started at http://127.11.118.194:39625/ using document root <none> and password file <none>
I20260929 18:44:30.783278 28987 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:30.783370 28987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:30.783599 28987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:30.785619 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "664005c4db53465dbb8ca3f34e1ec6ef"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "406c98cb0c6551b70d008d7c94337ed8"
server_key_iv: "211fde07b2c8bbe81c090a5eeb51a98b"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.786201 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "664005c4db53465dbb8ca3f34e1ec6ef"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "406c98cb0c6551b70d008d7c94337ed8"
server_key_iv: "211fde07b2c8bbe81c090a5eeb51a98b"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.789968 28987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:30.792485 29002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.793702 28987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 18:44:30.793870 28987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "664005c4db53465dbb8ca3f34e1ec6ef"
format_stamp: "Formatted at 2026-09-29 18:44:30 on dist-test-slave-cgv7"
server_key: "406c98cb0c6551b70d008d7c94337ed8"
server_key_iv: "211fde07b2c8bbe81c090a5eeb51a98b"
server_key_version: "encryptionkey@0"
I20260929 18:44:30.793995 28987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:30.805976 28987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:30.809437 28987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:30.809681 28987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:30.810401 28987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:30.811292 28987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:30.811367 28987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.811436 28987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:30.811489 28987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:30.820866 28987 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:42857
I20260929 18:44:30.820920 29115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:42857 every 8 connection(s)
I20260929 18:44:30.821861 28987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:30.828428 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 28987
I20260929 18:44:30.828573 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:30.828908 11739 external_mini_cluster.cc:1428] Setting key 6a46b2e1264f7b9d272aa756be1954f2
Sep 29 18:44:30 dist-test-slave-cgv7 krb5kdc[28746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:30.837257 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:30.823857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:41399 (local address 127.11.118.254:36617)
0929 18:44:30.824006 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:30.824010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:30.824817 (+ 807us) server_negotiation.cc:407] Connection header received
0929 18:44:30.826098 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:30.826101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:30.826151 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:30.826228 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:30.827488 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.828003 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.829742 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.830051 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.830179 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:30.833486 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:30.833490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:30.833493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:30.833526 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:30.835033 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.835637 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.835641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.835642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.835736 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:30.836099 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:30.836104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:30.836106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:30.836312 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:30.836431 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:30.836965 (+ 534us) server_negotiation.cc:299] Negotiation successful
0929 18:44:30.837093 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:30.838158 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:30.824175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36617 (local address 127.11.118.194:41399)
0929 18:44:30.824649 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:30.824695 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:30.825902 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:30.826489 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:30.826496 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:30.826849 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:30.827353 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:30.827368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.828155 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.828161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:30.829590 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:30.829601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:30.830225 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:30.830230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:30.830477 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:30.831328 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:30.831353 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:30.833294 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:30.835207 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:30.835213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:30.835237 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:30.835518 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:30.835842 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:30.835849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:30.835850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:30.835997 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:30.836429 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:30.836436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:30.836703 (+ 267us) client_negotiation.cc:769] Sending connection context
0929 18:44:30.836924 (+ 221us) client_negotiation.cc:240] Negotiation successful
0929 18:44:30.837274 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
I20260929 18:44:30.839592 29116 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36617
I20260929 18:44:30.839844 29116 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:30.840389 29116 heartbeater.cc:511] Master 127.11.118.254:36617 requested a full tablet report, sending...
I20260929 18:44:30.841530 28790 ts_manager.cc:194] Registered new tserver with Master: 664005c4db53465dbb8ca3f34e1ec6ef (127.11.118.194:42857)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:44:30.842303 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:41399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:44:30.858182 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:30.975237 29121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:30.975513 29121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:30.975557 29121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:30.979863 29121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:30.979934 29121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:30.980043 29121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:30.985652 29121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:30.986832 29121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:30.987910 29121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:30.997884 29126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:30.997942 29129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:30.997931 29121 server_base.cc:1034] running on GCE node
W20260929 18:44:30.997875 29127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:30.998641 29121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:30.999426 29121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:31.000649 29121 hybrid_clock.cc:648] HybridClock initialized: now 1790707471000613 us; error 40 us; skew 500 ppm
Sep 29 18:44:31 dist-test-slave-cgv7 krb5kdc[28746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707471, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:31.003667 29121 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:44:31.004635 29121 webserver.cc:460] Webserver started at http://127.11.118.195:39023/ using document root <none> and password file <none>
I20260929 18:44:31.005214 29121 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:31.005302 29121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:31.005545 29121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:31.007661 29121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "a0b4f2ab9b4abb4819defe28fc75af3d"
server_key_iv: "13d7b15768e1b88bf376880c62a38d9c"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.008237 29121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "a0b4f2ab9b4abb4819defe28fc75af3d"
server_key_iv: "13d7b15768e1b88bf376880c62a38d9c"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.012187 29121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:44:31.014720 29136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.016027 29121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:31.016177 29121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "a0b4f2ab9b4abb4819defe28fc75af3d"
server_key_iv: "13d7b15768e1b88bf376880c62a38d9c"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.016295 29121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:31.044010 29121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:31.047250 29121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:31.047472 29121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:31.048072 29121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:31.048923 29121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:31.048995 29121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.049065 29121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:31.049109 29121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.058457 29121 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:33121
I20260929 18:44:31.058475 29249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:33121 every 8 connection(s)
I20260929 18:44:31.059372 29121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:31.065567 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29121
I20260929 18:44:31.065729 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:31.066051 11739 external_mini_cluster.cc:1428] Setting key 8a9ed881b160916233f4d402d65f8517
Sep 29 18:44:31 dist-test-slave-cgv7 krb5kdc[28746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707471, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:31.074412 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.061153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:60037 (local address 127.11.118.254:36617)
0929 18:44:31.061285 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.061290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.062252 (+ 962us) server_negotiation.cc:407] Connection header received
0929 18:44:31.063587 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.063590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.063655 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.063739 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:31.064919 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.065606 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.067340 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.067639 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.067734 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.070482 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:31.070486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:31.070488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:31.070513 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:31.072188 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:31.072834 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:31.072838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:31.072840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:31.072936 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:31.073265 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:31.073267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:31.073268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:31.073453 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:31.073530 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.074090 (+ 560us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.074196 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:44:31.075201 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.061561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:36617 (local address 127.11.118.195:60037)
0929 18:44:31.062103 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.062148 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.063371 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.063917 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.063924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.064280 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:31.064757 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.064770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.065831 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.065835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.067129 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.067142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.067803 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.067807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.068013 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.068665 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:31.068681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:31.070249 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:31.072313 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:31.072319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:31.072334 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:31.072604 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:31.073027 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:31.073029 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:31.073031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:31.073171 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:31.073538 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:31.073543 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:31.073792 (+ 249us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.073991 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.074321 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":107,"threads_started":1}
I20260929 18:44:31.076586 29250 heartbeater.cc:347] Connected to a master server at 127.11.118.254:36617
I20260929 18:44:31.076924 29250 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:31.077445 29250 heartbeater.cc:511] Master 127.11.118.254:36617 requested a full tablet report, sending...
I20260929 18:44:31.078645 28790 ts_manager.cc:194] Registered new tserver with Master: 0fbd2c7c9afb4be9902fccb2d1b53f91 (127.11.118.195:33121)
I20260929 18:44:31.079277 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:60037
I20260929 18:44:31.080370 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:31.090939 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.082836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.11.118.254:36617)
0929 18:44:31.082998 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.083002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.083035 (+ 33us) server_negotiation.cc:407] Connection header received
0929 18:44:31.083219 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.083222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.083263 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.083326 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:31.084181 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.084743 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.085652 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.085889 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.085976 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.087421 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:31.087424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:31.087426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:31.087449 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:31.088970 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:31.089440 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:31.089443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:31.089444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:31.089531 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:31.089883 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:31.089886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:31.089887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:31.090139 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:31.090252 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.090577 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.090683 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 18:44:31.093900 28790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:31.096973 28790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:31.112320 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.106535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.11.118.194:42857)
0929 18:44:31.106875 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.106888 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.106917 (+ 29us) server_negotiation.cc:407] Connection header received
0929 18:44:31.107002 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.107008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.107223 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.107373 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:31.107911 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109135 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.110900 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.111599 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.111743 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.111887 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.112058 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.112164 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":131,"threads_started":1}
I20260929 18:44:31.112336 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.106019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:42857 (local address 127.0.0.1:57170)
0929 18:44:31.106497 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.106550 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.106689 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.107575 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.107585 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.107611 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:31.107767 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.107778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109300 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.109305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.110754 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.110764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.111728 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.111731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.111844 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.111881 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.112025 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.112157 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":95,"thread_start_us":80,"threads_started":1}
I20260929 18:44:31.112495 29263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.106134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.11.118.195:33121)
0929 18:44:31.106605 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.106617 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.107008 (+ 391us) server_negotiation.cc:407] Connection header received
0929 18:44:31.107338 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.107344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.107508 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.107629 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:31.108032 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109158 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.110844 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.111617 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.111713 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.111846 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.112223 (+ 377us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.112329 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":166,"threads_started":1}
I20260929 18:44:31.112927 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.106021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:33121 (local address 127.0.0.1:46490)
0929 18:44:31.106880 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.106896 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.107157 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.107790 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.107793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.107810 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:31.107913 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.107919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109268 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.109272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.110707 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.110719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.111962 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.111966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.112061 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.112083 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.112700 (+ 617us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.112782 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":158,"thread_start_us":120,"threads_started":1}
I20260929 18:44:31.112962 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.107049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:41947 (local address 127.0.0.1:49394)
0929 18:44:31.107846 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.107859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.108052 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.108504 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.108507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.108526 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:31.108631 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.108641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109899 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.109903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.111147 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.111156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.112377 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.112381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.112703 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.112729 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.112800 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.112857 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":56,"thread_start_us":108,"threads_started":1}
I20260929 18:44:31.113474 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.107219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.11.118.193:41947)
0929 18:44:31.107578 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.107592 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.108158 (+ 566us) server_negotiation.cc:407] Connection header received
0929 18:44:31.108223 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.108226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.108366 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.108527 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:31.108733 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.109739 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.111275 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.112087 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.112207 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.112318 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.113083 (+ 765us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.113310 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":85,"threads_started":1}
I20260929 18:44:31.115691 29050 tablet_service.cc:1467] Processing CreateTablet for tablet 731660c2212b46ab945984faa7391c62 (DEFAULT_TABLE table=test-table [id=f153022bf8314761900e3ddce3011baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:31.116102 29184 tablet_service.cc:1467] Processing CreateTablet for tablet 731660c2212b46ab945984faa7391c62 (DEFAULT_TABLE table=test-table [id=f153022bf8314761900e3ddce3011baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:31.116102 28916 tablet_service.cc:1467] Processing CreateTablet for tablet 731660c2212b46ab945984faa7391c62 (DEFAULT_TABLE table=test-table [id=f153022bf8314761900e3ddce3011baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:31.116782 29050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731660c2212b46ab945984faa7391c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:31.117118 28916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731660c2212b46ab945984faa7391c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:31.117118 29184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731660c2212b46ab945984faa7391c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:31.124795 29267 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Bootstrap starting.
I20260929 18:44:31.125139 29269 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: Bootstrap starting.
I20260929 18:44:31.125633 29268 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Bootstrap starting.
I20260929 18:44:31.127213 29267 tablet_bootstrap.cc:654] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:31.127588 29269 tablet_bootstrap.cc:654] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:31.127811 29268 tablet_bootstrap.cc:654] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:31.128400 29267 log.cc:824] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:31.128577 29269 log.cc:824] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:31.128979 29268 log.cc:824] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:31.130777 29267 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: No bootstrap required, opened a new log
I20260929 18:44:31.130987 29267 ts_tablet_manager.cc:1397] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:44:31.131209 29268 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: No bootstrap required, opened a new log
I20260929 18:44:31.131397 29268 ts_tablet_manager.cc:1397] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:31.131585 29269 tablet_bootstrap.cc:492] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: No bootstrap required, opened a new log
I20260929 18:44:31.131795 29269 ts_tablet_manager.cc:1397] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 18:44:31.134234 29267 raft_consensus.cc:348] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.134567 29267 raft_consensus.cc:374] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:31.134608 29267 raft_consensus.cc:729] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664005c4db53465dbb8ca3f34e1ec6ef, State: Initialized, Role: FOLLOWER
I20260929 18:44:31.135048 29267 consensus_queue.cc:260] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.135442 29268 raft_consensus.cc:348] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.135690 29268 raft_consensus.cc:374] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:31.135747 29268 raft_consensus.cc:729] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fbd2c7c9afb4be9902fccb2d1b53f91, State: Initialized, Role: FOLLOWER
I20260929 18:44:31.135773 29116 heartbeater.cc:503] Master 127.11.118.254:36617 was elected leader, sending a full tablet report...
I20260929 18:44:31.136118 29267 ts_tablet_manager.cc:1428] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260929 18:44:31.136175 29268 consensus_queue.cc:260] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.136111 29269 raft_consensus.cc:348] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.136363 29269 raft_consensus.cc:374] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:31.136433 29269 raft_consensus.cc:729] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6459b38c542f47508fb254c2ef60a3f9, State: Initialized, Role: FOLLOWER
I20260929 18:44:31.136886 29269 consensus_queue.cc:260] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.137197 29268 ts_tablet_manager.cc:1428] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:44:31.137531 29250 heartbeater.cc:503] Master 127.11.118.254:36617 was elected leader, sending a full tablet report...
I20260929 18:44:31.137817 29269 ts_tablet_manager.cc:1428] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:31.138226 28982 heartbeater.cc:503] Master 127.11.118.254:36617 was elected leader, sending a full tablet report...
I20260929 18:44:31.234798 29273 raft_consensus.cc:482] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:31.235031 29273 raft_consensus.cc:504] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.236452 29273 leader_election.cc:288] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fbd2c7c9afb4be9902fccb2d1b53f91 (127.11.118.195:33121), 6459b38c542f47508fb254c2ef60a3f9 (127.11.118.193:41947)
I20260929 18:44:31.241443 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.237147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:44261 (local address 127.11.118.193:41947)
0929 18:44:31.237285 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.237289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.237304 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:44:31.237438 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.237440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.237516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.237599 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:31.237868 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.238589 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240074 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.240720 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240888 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.240938 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.241226 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.241306 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:44:31.241443 29263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.236790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:37871 (local address 127.11.118.195:33121)
0929 18:44:31.236920 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.236925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.236943 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:44:31.237054 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.237060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.237118 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.237207 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:31.237545 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.238385 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240073 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.240720 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240856 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.240912 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.241226 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.241306 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:31.241475 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.236652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:33121 (local address 127.11.118.194:37871)
0929 18:44:31.236849 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.236867 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.236949 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.237234 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.237237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.237262 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:31.237412 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.237422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.238500 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.238503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.239900 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.239911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.240856 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.241044 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.241108 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.241188 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.241312 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":176}
I20260929 18:44:31.241484 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.236688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:41947 (local address 127.11.118.194:44261)
0929 18:44:31.237236 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.237248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.237327 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.237622 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.237625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.237640 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:31.237738 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.237742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.238687 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.238691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.239924 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.239930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.240856 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.240860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.241069 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.241111 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.241188 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.241316 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":123,"thread_start_us":144,"threads_started":1}
I20260929 18:44:31.242388 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731660c2212b46ab945984faa7391c62" candidate_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" is_pre_election: true
I20260929 18:44:31.242537 28936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731660c2212b46ab945984faa7391c62" candidate_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6459b38c542f47508fb254c2ef60a3f9" is_pre_election: true
I20260929 18:44:31.242750 29204 raft_consensus.cc:2454] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664005c4db53465dbb8ca3f34e1ec6ef in term 0.
I20260929 18:44:31.242897 28936 raft_consensus.cc:2454] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664005c4db53465dbb8ca3f34e1ec6ef in term 0.
I20260929 18:44:31.243306 29006 leader_election.cc:302] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0fbd2c7c9afb4be9902fccb2d1b53f91, 6459b38c542f47508fb254c2ef60a3f9, 664005c4db53465dbb8ca3f34e1ec6ef; no voters:
I20260929 18:44:31.243643 29273 raft_consensus.cc:2781] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:31.243743 29273 raft_consensus.cc:482] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:31.243801 29273 raft_consensus.cc:3037] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:31.244904 29273 raft_consensus.cc:504] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.245280 29273 leader_election.cc:288] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [CANDIDATE]: Term 1 election: Requested vote from peers 0fbd2c7c9afb4be9902fccb2d1b53f91 (127.11.118.195:33121), 6459b38c542f47508fb254c2ef60a3f9 (127.11.118.193:41947)
I20260929 18:44:31.245721 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731660c2212b46ab945984faa7391c62" candidate_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91"
I20260929 18:44:31.245725 28936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731660c2212b46ab945984faa7391c62" candidate_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6459b38c542f47508fb254c2ef60a3f9"
I20260929 18:44:31.245913 29204 raft_consensus.cc:3037] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:31.245870 28936 raft_consensus.cc:3037] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:31.247339 29204 raft_consensus.cc:2454] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 664005c4db53465dbb8ca3f34e1ec6ef in term 1.
I20260929 18:44:31.247360 28936 raft_consensus.cc:2454] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 664005c4db53465dbb8ca3f34e1ec6ef in term 1.
I20260929 18:44:31.247671 29006 leader_election.cc:302] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0fbd2c7c9afb4be9902fccb2d1b53f91, 6459b38c542f47508fb254c2ef60a3f9, 664005c4db53465dbb8ca3f34e1ec6ef; no voters:
I20260929 18:44:31.247886 29273 raft_consensus.cc:2781] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:31.248070 29273 raft_consensus.cc:686] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 1 LEADER]: Becoming Leader. State: Replica: 664005c4db53465dbb8ca3f34e1ec6ef, State: Running, Role: LEADER
I20260929 18:44:31.248405 29273 consensus_queue.cc:237] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } }
I20260929 18:44:31.251799 28791 catalog_manager.cc:5495] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 664005c4db53465dbb8ca3f34e1ec6ef (127.11.118.194). New cstate: current_term: 1 leader_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "664005c4db53465dbb8ca3f34e1ec6ef" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 } health_report { overall_health: UNKNOWN } } }
I20260929 18:44:31.275173 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.271687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.11.118.194:42857)
0929 18:44:31.271828 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.271832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.271846 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:44:31.271984 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.271987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.272045 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.272134 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:31.272363 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.272861 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.273711 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.273919 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.273992 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.274284 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:31.274735 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:31.274833 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.274931 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.274986 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 18:44:31.281418 29204 raft_consensus.cc:1261] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 1 FOLLOWER]: Refusing update from remote peer 664005c4db53465dbb8ca3f34e1ec6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:31.281418 28936 raft_consensus.cc:1261] T 731660c2212b46ab945984faa7391c62 P 6459b38c542f47508fb254c2ef60a3f9 [term 1 FOLLOWER]: Refusing update from remote peer 664005c4db53465dbb8ca3f34e1ec6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:31.282333 29273 consensus_queue.cc:1035] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fbd2c7c9afb4be9902fccb2d1b53f91" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 33121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:31.282582 29277 consensus_queue.cc:1035] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6459b38c542f47508fb254c2ef60a3f9" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:31.291046 29281 mvcc.cc:205] Tried to move back new op lower bound from 7334737802360410112 to 7334737802236215296. Current Snapshot: MvccSnapshot[applied={T|T < 7334737802360410112}]
I20260929 18:44:31.291364 29282 mvcc.cc:205] Tried to move back new op lower bound from 7334737802360410112 to 7334737802236215296. Current Snapshot: MvccSnapshot[applied={T|T < 7334737802360410112}]
I20260929 18:44:31.299373 28790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:31.299578 28790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:31.302198 28790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b915267b0824c148562cc6ebfa9765c: Sending DeleteTablet for 3 replicas of tablet 731660c2212b46ab945984faa7391c62
I20260929 18:44:31.303180 29184 tablet_service.cc:1514] Processing DeleteTablet for tablet 731660c2212b46ab945984faa7391c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:46490
I20260929 18:44:31.303241 28916 tablet_service.cc:1514] Processing DeleteTablet for tablet 731660c2212b46ab945984faa7391c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49394
I20260929 18:44:31.303435 29050 tablet_service.cc:1514] Processing DeleteTablet for tablet 731660c2212b46ab945984faa7391c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:31 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:57170
I20260929 18:44:31.303920 29292 tablet_replica.cc:330] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: stopping tablet replica
I20260929 18:44:31.304059 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28853
I20260929 18:44:31.305927 29292 raft_consensus.cc:2229] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:44:31.306382 29292 raft_consensus.cc:2258] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:31.307861 29292 ts_tablet_manager.cc:1905] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:31.310909 29292 ts_tablet_manager.cc:1918] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:31.311021 29292 log.cc:1196] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/731660c2212b46ab945984faa7391c62
I20260929 18:44:31.311240 29291 tablet_replica.cc:330] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: stopping tablet replica
I20260929 18:44:31.311327 29292 ts_tablet_manager.cc:1939] T 731660c2212b46ab945984faa7391c62 P 664005c4db53465dbb8ca3f34e1ec6ef: Deleting consensus metadata
I20260929 18:44:31.311666 29291 raft_consensus.cc:2229] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:31.312052 29291 raft_consensus.cc:2258] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:31.312683 28776 catalog_manager.cc:4841] TS 664005c4db53465dbb8ca3f34e1ec6ef (127.11.118.194:42857): tablet 731660c2212b46ab945984faa7391c62 (table test-table [id=f153022bf8314761900e3ddce3011baf]) successfully deleted
I20260929 18:44:31.313679 29291 ts_tablet_manager.cc:1905] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:31.316425 29291 ts_tablet_manager.cc:1918] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:31.316530 29291 log.cc:1196] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/731660c2212b46ab945984faa7391c62
I20260929 18:44:31.316852 29291 ts_tablet_manager.cc:1939] T 731660c2212b46ab945984faa7391c62 P 0fbd2c7c9afb4be9902fccb2d1b53f91: Deleting consensus metadata
I20260929 18:44:31.318405 28778 catalog_manager.cc:4841] TS 0fbd2c7c9afb4be9902fccb2d1b53f91 (127.11.118.195:33121): tablet 731660c2212b46ab945984faa7391c62 (table test-table [id=f153022bf8314761900e3ddce3011baf]) successfully deleted
I20260929 18:44:31.322579 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28987
W20260929 18:44:31.322664 28776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:41947 (error 108)
I20260929 18:44:31.324621 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.324341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:41947 (local address 127.0.0.1:49410)
0929 18:44:31.324466 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.324538 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:41947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 18:44:31.324841 28776 catalog_manager.cc:4584] TS 6459b38c542f47508fb254c2ef60a3f9 (127.11.118.193:41947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 731660c2212b46ab945984faa7391c62: Network error: Client connection negotiation failed: client connection to 127.11.118.193:41947: connect: Connection refused (error 111)
I20260929 18:44:31.329831 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29121
I20260929 18:44:31.336098 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 28759
2026-09-29T18:44:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2428 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T18:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:31Z Disabled control of system clock
I20260929 18:44:31.367300 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38997
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:38997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:31.480787 29297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:31.481082 29297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:31.481118 29297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:31.485219 29297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:31.485287 29297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:31.485302 29297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:31.485318 29297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:31.485333 29297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:31.490341 29297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:38997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:38997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:31.491439 29297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:31.492797 29297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:31.502008 29302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:31.502008 29305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:31.502090 29297 server_base.cc:1034] running on GCE node
W20260929 18:44:31.502008 29303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:31.502885 29297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:31.504168 29297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:31.505388 29297 hybrid_clock.cc:648] HybridClock initialized: now 1790707471505373 us; error 32 us; skew 500 ppm
I20260929 18:44:31.507613 29297 webserver.cc:460] Webserver started at http://127.11.118.254:36999/ using document root <none> and password file <none>
I20260929 18:44:31.508239 29297 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:31.508338 29297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:31.508584 29297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:31.510815 29297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "5d00d2db811240699db4807f3b631d20"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "c4ed96b8c44f986373d25b98e92640af"
server_key_iv: "7a201bc6c93b81a84eac10be16e440d7"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.511422 29297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "5d00d2db811240699db4807f3b631d20"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "c4ed96b8c44f986373d25b98e92640af"
server_key_iv: "7a201bc6c93b81a84eac10be16e440d7"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.515396 29297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:44:31.518028 29311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.519447 29297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:31.519610 29297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "5d00d2db811240699db4807f3b631d20"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "c4ed96b8c44f986373d25b98e92640af"
server_key_iv: "7a201bc6c93b81a84eac10be16e440d7"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.519743 29297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:31.536604 29297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:31.537362 29297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:31.537567 29297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:31.544660 29297 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:38997
I20260929 18:44:31.544664 29363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:38997 every 8 connection(s)
I20260929 18:44:31.545670 29297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:31.548857 29364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:31.554337 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29297
I20260929 18:44:31.554512 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:31.554658 29364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: Bootstrap starting.
I20260929 18:44:31.554829 11739 external_mini_cluster.cc:1428] Setting key eec7bc92ee65b24959f871b2c30c6a85
I20260929 18:44:31.557528 29364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:31.558490 29364 log.cc:824] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:31.560927 29364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: No bootstrap required, opened a new log
I20260929 18:44:31.565758 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.556297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.11.118.254:38997)
0929 18:44:31.557021 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.557040 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.557082 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:44:31.558062 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.558086 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.558641 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.559023 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:31.559447 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.560858 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.561825 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.562556 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.562709 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.563571 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:31.563577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:31.563595 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:31.564116 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:31.564191 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:31.564217 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:31.564336 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.564447 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.564792 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":118,"threads_started":1}
I20260929 18:44:31.565644 29364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } }
I20260929 18:44:31.565917 29364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:31.565984 29364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d00d2db811240699db4807f3b631d20, State: Initialized, Role: FOLLOWER
I20260929 18:44:31.566465 29364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } }
I20260929 18:44:31.566581 29364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:31.566622 29364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:31.566684 29364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:31.567811 29364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } }
I20260929 18:44:31.568188 29364 leader_election.cc:302] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d00d2db811240699db4807f3b631d20; no voters:
I20260929 18:44:31.568511 29364 leader_election.cc:288] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:31.568643 29369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:31.568907 29369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [term 1 LEADER]: Becoming Leader. State: Replica: 5d00d2db811240699db4807f3b631d20, State: Running, Role: LEADER
I20260929 18:44:31.569196 29369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } }
I20260929 18:44:31.569896 29364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:31.571010 29370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d00d2db811240699db4807f3b631d20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } } }
I20260929 18:44:31.571138 29370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:31.571585 29378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:31.571524 29371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d00d2db811240699db4807f3b631d20. Latest consensus state: current_term: 1 leader_uuid: "5d00d2db811240699db4807f3b631d20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d00d2db811240699db4807f3b631d20" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 38997 } } }
I20260929 18:44:31.571652 29371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:31.574887 29378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:31.581453 29378 catalog_manager.cc:1347] Generated new cluster ID: 111ae15184e44c71beda14bddfd11552
I20260929 18:44:31.581570 29378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:31.596438 29378 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:31.597473 29378 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:31.602126 29378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: Generated new TSK 0
I20260929 18:44:31.602994 29378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:31.613178 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--builtin_ntp_servers=127.11.118.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:31.731649 29388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:31.731932 29388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:31.731974 29388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:31.736126 29388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:31.736187 29388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:31.736248 29388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:31.741102 29388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:31.742208 29388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:31.743628 29388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:31.753516 29393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:31.753499 29394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:31.753492 29396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:31.753933 29388 server_base.cc:1034] running on GCE node
I20260929 18:44:31.754420 29388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:31.755127 29388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:31.756304 29388 hybrid_clock.cc:648] HybridClock initialized: now 1790707471756284 us; error 34 us; skew 500 ppm
I20260929 18:44:31.758437 29388 webserver.cc:460] Webserver started at http://127.11.118.193:38423/ using document root <none> and password file <none>
I20260929 18:44:31.759057 29388 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:31.759124 29388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:31.759344 29388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:31.761457 29388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "79edaf74477d4b43a71f5e1b9a26d92f"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "6baeda4174418b430d2c661c6345e10c"
server_key_iv: "40647f451ccf68849a85835f4b533cf2"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.762030 29388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "79edaf74477d4b43a71f5e1b9a26d92f"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "6baeda4174418b430d2c661c6345e10c"
server_key_iv: "40647f451ccf68849a85835f4b533cf2"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.765923 29388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 18:44:31.768543 29402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.769984 29388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:44:31.770113 29388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "79edaf74477d4b43a71f5e1b9a26d92f"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "6baeda4174418b430d2c661c6345e10c"
server_key_iv: "40647f451ccf68849a85835f4b533cf2"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.770231 29388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:31.784489 29388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:31.785183 29388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:31.785386 29388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:31.785975 29388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:31.786965 29388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:31.787036 29388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.787106 29388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:31.787154 29388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.796059 29388 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:37715
I20260929 18:44:31.796083 29515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:37715 every 8 connection(s)
I20260929 18:44:31.796986 29388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:31.800021 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29388
I20260929 18:44:31.800199 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:31.800643 11739 external_mini_cluster.cc:1428] Setting key 4184f06b5e6ba16927064c36496fcb26
I20260929 18:44:31.802650 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--builtin_ntp_servers=127.11.118.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:31.807790 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.798893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:50553 (local address 127.11.118.254:38997)
0929 18:44:31.799064 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:31.799068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:31.800381 (+ 1313us) server_negotiation.cc:407] Connection header received
0929 18:44:31.801932 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:31.801935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:31.801988 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:31.802073 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:31.803147 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.804008 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.805732 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.805974 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.806090 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.807275 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:31.807280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:31.807282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:31.807304 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:31.807322 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:31.807329 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:31.807405 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:31.807573 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 18:44:31.807636 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 18:44:31.808655 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:31.799269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38997 (local address 127.11.118.193:50553)
0929 18:44:31.800139 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:31.800206 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:31.801722 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:31.802255 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:31.802312 (+ 57us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:31.802410 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:31.802988 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.803008 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.804163 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.804169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.805608 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:31.805618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:31.806132 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:31.806136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:31.806362 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:31.807002 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:31.807003 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:31.807044 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:31.807111 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:31.807419 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:31.807431 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:31.807440 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 18:44:31.807561 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 18:44:31.807851 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":152,"threads_started":1}
I20260929 18:44:31.810027 29516 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38997
I20260929 18:44:31.810328 29516 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:31.810878 29516 heartbeater.cc:511] Master 127.11.118.254:38997 requested a full tablet report, sending...
I20260929 18:44:31.812621 29328 ts_manager.cc:194] Registered new tserver with Master: 79edaf74477d4b43a71f5e1b9a26d92f (127.11.118.193:37715)
I20260929 18:44:31.814373 29328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.193:50553
W20260929 18:44:31.921303 29519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:31.921593 29519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:31.921633 29519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:31.926108 29519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:31.926182 29519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:31.926241 29519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:31.931371 29519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:31.932469 29519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:31.933804 29519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:31.943243 29525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:31.943361 29519 server_base.cc:1034] running on GCE node
W20260929 18:44:31.943310 29527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:31.943241 29524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:31.943898 29519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:31.944583 29519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:31.945770 29519 hybrid_clock.cc:648] HybridClock initialized: now 1790707471945719 us; error 63 us; skew 500 ppm
I20260929 18:44:31.947681 29519 webserver.cc:460] Webserver started at http://127.11.118.194:36257/ using document root <none> and password file <none>
I20260929 18:44:31.948263 29519 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:31.948356 29519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:31.948585 29519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:31.950868 29519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "eb570610b44e4db2b15c90e7a8fc4526"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "3a760fa9403e12ff276222ef562681ca"
server_key_iv: "74579ebdb9a38200f4a18401b67aa83e"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.951442 29519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "eb570610b44e4db2b15c90e7a8fc4526"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "3a760fa9403e12ff276222ef562681ca"
server_key_iv: "74579ebdb9a38200f4a18401b67aa83e"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.955178 29519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 18:44:31.957664 29533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.959084 29519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:44:31.959223 29519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "eb570610b44e4db2b15c90e7a8fc4526"
format_stamp: "Formatted at 2026-09-29 18:44:31 on dist-test-slave-cgv7"
server_key: "3a760fa9403e12ff276222ef562681ca"
server_key_iv: "74579ebdb9a38200f4a18401b67aa83e"
server_key_version: "encryptionkey@0"
I20260929 18:44:31.959342 29519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:31.989637 29519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:31.990439 29519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:31.990653 29519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:31.991267 29519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:31.992216 29519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:31.992287 29519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:31.992354 29519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:31.992408 29519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:32.001065 29519 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:39395
I20260929 18:44:32.001084 29646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:39395 every 8 connection(s)
I20260929 18:44:32.001976 29519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:32.010388 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29519
I20260929 18:44:32.010569 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:32.010977 11739 external_mini_cluster.cc:1428] Setting key 105c25836a1438d50d4808c57c0cabe0
I20260929 18:44:32.011206 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.003793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:57909 (local address 127.11.118.254:38997)
0929 18:44:32.003948 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.003951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.005005 (+ 1054us) server_negotiation.cc:407] Connection header received
0929 18:44:32.006369 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.006374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.006428 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.006504 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:32.007398 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.007922 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.009015 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.009217 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.009291 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.010588 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:32.010592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:32.010595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:32.010612 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:32.010638 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:32.010649 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:32.010730 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.010956 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.011013 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 18:44:32.012178 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.004222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38997 (local address 127.11.118.194:57909)
0929 18:44:32.004853 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.004893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.006118 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.006664 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.006670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.006752 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:32.007256 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.007268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.008027 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.008030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.008908 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.008916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.009344 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.009347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.009495 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.010147 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:32.010151 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:32.010210 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:32.010347 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:32.010755 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:32.010765 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:32.010773 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.010915 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.011228 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260929 18:44:32.013118 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--builtin_ntp_servers=127.11.118.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:32.013356 29647 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38997
I20260929 18:44:32.013597 29647 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:32.014202 29647 heartbeater.cc:511] Master 127.11.118.254:38997 requested a full tablet report, sending...
I20260929 18:44:32.015388 29328 ts_manager.cc:194] Registered new tserver with Master: eb570610b44e4db2b15c90e7a8fc4526 (127.11.118.194:39395)
I20260929 18:44:32.015975 29328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.194:57909
W20260929 18:44:32.126478 29650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:32.126827 29650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:32.126909 29650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:32.131099 29650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:32.131186 29650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:32.131276 29650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:32.136189 29650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:38997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:32.137346 29650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:32.138828 29650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:32.148643 29658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:32.148638 29655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:32.148638 29656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:32.148958 29650 server_base.cc:1034] running on GCE node
I20260929 18:44:32.149477 29650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:32.150161 29650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:32.151353 29650 hybrid_clock.cc:648] HybridClock initialized: now 1790707472151315 us; error 48 us; skew 500 ppm
I20260929 18:44:32.153216 29650 webserver.cc:460] Webserver started at http://127.11.118.195:34283/ using document root <none> and password file <none>
I20260929 18:44:32.153805 29650 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:32.153936 29650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:32.154176 29650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:32.156312 29650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "2647469dd38546e9aca7b7e4995aa034"
format_stamp: "Formatted at 2026-09-29 18:44:32 on dist-test-slave-cgv7"
server_key: "9a3abb0b9a44bcd33adf2ce36755e6c3"
server_key_iv: "bee42b8eefa957bd39f5cd5b1c14840d"
server_key_version: "encryptionkey@0"
I20260929 18:44:32.156886 29650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "2647469dd38546e9aca7b7e4995aa034"
format_stamp: "Formatted at 2026-09-29 18:44:32 on dist-test-slave-cgv7"
server_key: "9a3abb0b9a44bcd33adf2ce36755e6c3"
server_key_iv: "bee42b8eefa957bd39f5cd5b1c14840d"
server_key_version: "encryptionkey@0"
I20260929 18:44:32.160799 29650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:32.163427 29664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:32.164748 29650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:32.164903 29650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "2647469dd38546e9aca7b7e4995aa034"
format_stamp: "Formatted at 2026-09-29 18:44:32 on dist-test-slave-cgv7"
server_key: "9a3abb0b9a44bcd33adf2ce36755e6c3"
server_key_iv: "bee42b8eefa957bd39f5cd5b1c14840d"
server_key_version: "encryptionkey@0"
I20260929 18:44:32.165023 29650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:32.176036 29650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:32.176721 29650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:32.176930 29650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:32.177515 29650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:32.178511 29650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:32.178580 29650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:32.178653 29650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:32.178694 29650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:32.187398 29650 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:37581
I20260929 18:44:32.187458 29777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:37581 every 8 connection(s)
I20260929 18:44:32.188359 29650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:32.189895 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29650
I20260929 18:44:32.190034 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:32.190405 11739 external_mini_cluster.cc:1428] Setting key b0109121b06e96f910f506c94d7fcce9
I20260929 18:44:32.198133 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.190396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:36559 (local address 127.11.118.254:38997)
0929 18:44:32.190570 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.190574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.191952 (+ 1378us) server_negotiation.cc:407] Connection header received
0929 18:44:32.193540 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.193543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.193589 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.193657 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:32.194680 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.195204 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.196335 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.196545 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.196622 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.197694 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:32.197697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:32.197699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:32.197712 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:32.197729 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:32.197736 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:32.197794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.197960 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.198011 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 18:44:32.199100 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.190888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:38997 (local address 127.11.118.195:36559)
0929 18:44:32.191759 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.191817 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.193333 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.193827 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.193834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.193952 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:32.194540 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.194556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.195311 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.195313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.196222 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.196231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.196675 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.196678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.196824 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.197444 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:32.197446 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:32.197486 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:32.197552 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:32.197808 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:32.197814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:32.197819 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.197939 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.198175 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":146,"threads_started":1}
I20260929 18:44:32.200364 29778 heartbeater.cc:347] Connected to a master server at 127.11.118.254:38997
I20260929 18:44:32.200582 29778 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:32.201085 29778 heartbeater.cc:511] Master 127.11.118.254:38997 requested a full tablet report, sending...
I20260929 18:44:32.202157 29328 ts_manager.cc:194] Registered new tserver with Master: 2647469dd38546e9aca7b7e4995aa034 (127.11.118.195:37581)
I20260929 18:44:32.202821 29328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.195:36559
I20260929 18:44:32.204592 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:32.210594 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.206633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.11.118.254:38997)
0929 18:44:32.206839 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.206845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.206940 (+ 95us) server_negotiation.cc:407] Connection header received
0929 18:44:32.207093 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.207096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.207142 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.207226 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:32.207519 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.208048 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.208891 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.209109 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.209221 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.210097 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:32.210103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:32.210105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:32.210116 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:32.210136 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:32.210144 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:32.210223 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.210386 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.210442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 18:44:32.213428 29328 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:32.216570 29328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:32.231673 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.225825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:39395 (local address 127.0.0.1:44294)
0929 18:44:32.226435 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.226447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.226531 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.227067 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.227073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.227092 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:32.227204 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.227210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228852 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.228855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.230047 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.230057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.231150 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.231152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.231232 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.231263 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.231377 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.231499 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":207,"thread_start_us":114,"threads_started":1}
I20260929 18:44:32.231679 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.226011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.11.118.194:39395)
0929 18:44:32.226640 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.226658 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.226678 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:44:32.226738 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.226741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.226918 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.227069 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:32.227338 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228547 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.230179 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.231027 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.231162 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.231357 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.231410 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.231508 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":137,"threads_started":1}
I20260929 18:44:32.231732 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.225156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:37581 (local address 127.0.0.1:51170)
0929 18:44:32.226017 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.226031 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.226115 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.226954 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.226960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.226989 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:32.227198 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.227210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228715 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.228718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.230367 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.230379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.231412 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.231415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.231491 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.231510 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.231569 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.231621 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":92,"thread_start_us":114,"threads_started":1}
I20260929 18:44:32.231889 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.226034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.11.118.195:37581)
0929 18:44:32.226480 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.226493 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.226519 (+ 26us) server_negotiation.cc:407] Connection header received
0929 18:44:32.226601 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.226607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.226790 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.226922 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:32.227381 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228547 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.230513 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.231269 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.231408 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.231558 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.231605 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.231692 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260929 18:44:32.234810 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.225089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37715 (local address 127.0.0.1:34956)
0929 18:44:32.225431 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.225478 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.225600 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.227605 (+ 2005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.227608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.227625 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:32.227713 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.227720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228727 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.228733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.230368 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.230379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.234389 (+ 4010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.234392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.234461 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.234479 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.234581 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.234654 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":186,"thread_start_us":73,"threads_started":1}
I20260929 18:44:32.235086 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.225202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.11.118.193:37715)
0929 18:44:32.227226 (+ 2024us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.227239 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.227262 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:44:32.227335 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.227339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.227482 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.227627 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:32.227841 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.228572 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.232073 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.232858 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.232996 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.233161 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.234615 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.234722 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1920,"thread_start_us":83,"threads_started":1}
I20260929 18:44:32.235183 29581 tablet_service.cc:1467] Processing CreateTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 (DEFAULT_TABLE table=test-table [id=9d4beeff11b6454f833e85621775f615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:32.235183 29712 tablet_service.cc:1467] Processing CreateTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 (DEFAULT_TABLE table=test-table [id=9d4beeff11b6454f833e85621775f615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:32.236342 29712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a64b379ba8543c596617596a6fdf3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:32.236342 29581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a64b379ba8543c596617596a6fdf3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:32.237514 29450 tablet_service.cc:1467] Processing CreateTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 (DEFAULT_TABLE table=test-table [id=9d4beeff11b6454f833e85621775f615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:32.238543 29450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a64b379ba8543c596617596a6fdf3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:32.243783 29796 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Bootstrap starting.
I20260929 18:44:32.244203 29795 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Bootstrap starting.
I20260929 18:44:32.245332 29797 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Bootstrap starting.
I20260929 18:44:32.246559 29795 tablet_bootstrap.cc:654] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:32.247402 29797 tablet_bootstrap.cc:654] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:32.247422 29795 log.cc:824] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:32.248034 29796 tablet_bootstrap.cc:654] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:32.248167 29797 log.cc:824] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:32.248802 29796 log.cc:824] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:32.249524 29795 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: No bootstrap required, opened a new log
I20260929 18:44:32.249725 29795 ts_tablet_manager.cc:1397] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:32.250554 29797 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: No bootstrap required, opened a new log
I20260929 18:44:32.250707 29796 tablet_bootstrap.cc:492] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: No bootstrap required, opened a new log
I20260929 18:44:32.250723 29797 ts_tablet_manager.cc:1397] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:32.250906 29796 ts_tablet_manager.cc:1397] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 18:44:32.254159 29796 raft_consensus.cc:348] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.254158 29795 raft_consensus.cc:348] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.254467 29796 raft_consensus.cc:374] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:32.254467 29795 raft_consensus.cc:374] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:32.254522 29796 raft_consensus.cc:729] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb570610b44e4db2b15c90e7a8fc4526, State: Initialized, Role: FOLLOWER
I20260929 18:44:32.254531 29795 raft_consensus.cc:729] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2647469dd38546e9aca7b7e4995aa034, State: Initialized, Role: FOLLOWER
I20260929 18:44:32.254801 29797 raft_consensus.cc:348] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.255053 29797 raft_consensus.cc:374] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:32.254987 29795 consensus_queue.cc:260] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.254981 29796 consensus_queue.cc:260] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.255105 29797 raft_consensus.cc:729] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79edaf74477d4b43a71f5e1b9a26d92f, State: Initialized, Role: FOLLOWER
I20260929 18:44:32.255504 29797 consensus_queue.cc:260] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.256076 29795 ts_tablet_manager.cc:1428] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:44:32.256376 29797 ts_tablet_manager.cc:1428] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:32.256444 29778 heartbeater.cc:503] Master 127.11.118.254:38997 was elected leader, sending a full tablet report...
I20260929 18:44:32.256680 29647 heartbeater.cc:503] Master 127.11.118.254:38997 was elected leader, sending a full tablet report...
I20260929 18:44:32.256711 29516 heartbeater.cc:503] Master 127.11.118.254:38997 was elected leader, sending a full tablet report...
I20260929 18:44:32.257596 29796 ts_tablet_manager.cc:1428] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260929 18:44:32.298190 29517 tablet.cc:2369] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:32.403906 29801 raft_consensus.cc:482] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:32.404189 29801 raft_consensus.cc:504] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.405511 29801 leader_election.cc:288] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79edaf74477d4b43a71f5e1b9a26d92f (127.11.118.193:37715), eb570610b44e4db2b15c90e7a8fc4526 (127.11.118.194:39395)
I20260929 18:44:32.410851 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.405955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:59119 (local address 127.11.118.194:39395)
0929 18:44:32.406106 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.406111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.406427 (+ 316us) server_negotiation.cc:407] Connection header received
0929 18:44:32.406603 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.406609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.406663 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.406731 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:32.407016 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.407733 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.409382 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.410022 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.410158 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.410212 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.410624 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.410710 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 18:44:32.410847 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.406124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:54219 (local address 127.11.118.193:37715)
0929 18:44:32.406240 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.406243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.406256 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:44:32.406324 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.406326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.406426 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.406507 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:32.406836 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.407635 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.409382 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.410022 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.410158 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.410212 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.410624 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.410710 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 18:44:32.410849 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.405715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37715 (local address 127.11.118.195:54219)
0929 18:44:32.405917 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.405938 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.406044 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.406547 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.406552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.406572 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:32.406683 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.406688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.407752 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.407755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.409221 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.409233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.410205 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.410209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.410430 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.410485 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.410571 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.410677 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":217}
I20260929 18:44:32.410866 29804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.405793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:39395 (local address 127.11.118.195:59119)
0929 18:44:32.406347 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.406360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:32.406462 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:32.406771 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:32.406776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:32.406796 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:32.406913 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.406918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.407837 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.407840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.409221 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:32.409233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.410205 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.410209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:32.410427 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.410480 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 18:44:32.410571 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 18:44:32.410677 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":131,"thread_start_us":182,"threads_started":1}
I20260929 18:44:32.411685 29601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a64b379ba8543c596617596a6fdf3a0" candidate_uuid: "2647469dd38546e9aca7b7e4995aa034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb570610b44e4db2b15c90e7a8fc4526" is_pre_election: true
I20260929 18:44:32.411676 29470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a64b379ba8543c596617596a6fdf3a0" candidate_uuid: "2647469dd38546e9aca7b7e4995aa034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" is_pre_election: true
I20260929 18:44:32.412050 29601 raft_consensus.cc:2454] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2647469dd38546e9aca7b7e4995aa034 in term 0.
I20260929 18:44:32.412050 29470 raft_consensus.cc:2454] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2647469dd38546e9aca7b7e4995aa034 in term 0.
I20260929 18:44:32.412680 29668 leader_election.cc:302] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2647469dd38546e9aca7b7e4995aa034, 79edaf74477d4b43a71f5e1b9a26d92f, eb570610b44e4db2b15c90e7a8fc4526; no voters:
I20260929 18:44:32.413025 29801 raft_consensus.cc:2781] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:32.413122 29801 raft_consensus.cc:482] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:32.413184 29801 raft_consensus.cc:3037] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:32.414460 29801 raft_consensus.cc:504] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.414889 29801 leader_election.cc:288] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [CANDIDATE]: Term 1 election: Requested vote from peers 79edaf74477d4b43a71f5e1b9a26d92f (127.11.118.193:37715), eb570610b44e4db2b15c90e7a8fc4526 (127.11.118.194:39395)
I20260929 18:44:32.415228 29601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a64b379ba8543c596617596a6fdf3a0" candidate_uuid: "2647469dd38546e9aca7b7e4995aa034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb570610b44e4db2b15c90e7a8fc4526"
I20260929 18:44:32.415390 29601 raft_consensus.cc:3037] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:32.416404 29601 raft_consensus.cc:2454] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2647469dd38546e9aca7b7e4995aa034 in term 1.
I20260929 18:44:32.416705 29470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a64b379ba8543c596617596a6fdf3a0" candidate_uuid: "2647469dd38546e9aca7b7e4995aa034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79edaf74477d4b43a71f5e1b9a26d92f"
I20260929 18:44:32.416790 29668 leader_election.cc:302] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2647469dd38546e9aca7b7e4995aa034, eb570610b44e4db2b15c90e7a8fc4526; no voters:
I20260929 18:44:32.416855 29470 raft_consensus.cc:3037] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:32.416960 29801 raft_consensus.cc:2781] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:32.417194 29801 raft_consensus.cc:686] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 1 LEADER]: Becoming Leader. State: Replica: 2647469dd38546e9aca7b7e4995aa034, State: Running, Role: LEADER
I20260929 18:44:32.417781 29801 consensus_queue.cc:237] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } }
I20260929 18:44:32.418133 29470 raft_consensus.cc:2454] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2647469dd38546e9aca7b7e4995aa034 in term 1.
I20260929 18:44:32.421396 29327 catalog_manager.cc:5495] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2647469dd38546e9aca7b7e4995aa034 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "2647469dd38546e9aca7b7e4995aa034" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2647469dd38546e9aca7b7e4995aa034" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 37581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 } health_report { overall_health: UNKNOWN } } }
W20260929 18:44:32.439388 29779 tablet.cc:2369] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:32.469323 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.465502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.11.118.195:37581)
0929 18:44:32.465634 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:32.465638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:32.465651 (+ 13us) server_negotiation.cc:407] Connection header received
0929 18:44:32.465737 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:32.465739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:32.465798 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:32.465902 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:32.466179 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.466875 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.467753 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:32.467973 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:32.468057 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:32.468395 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:32.468973 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:32.469065 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:32.469157 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 18:44:32.469210 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:44:32.475734 29601 raft_consensus.cc:1261] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 1 FOLLOWER]: Refusing update from remote peer 2647469dd38546e9aca7b7e4995aa034: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:32.475781 29470 raft_consensus.cc:1261] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 1 FOLLOWER]: Refusing update from remote peer 2647469dd38546e9aca7b7e4995aa034: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:32.476477 29805 consensus_queue.cc:1035] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb570610b44e4db2b15c90e7a8fc4526" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:32.476691 29801 consensus_queue.cc:1035] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79edaf74477d4b43a71f5e1b9a26d92f" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:32.485432 29809 mvcc.cc:205] Tried to move back new op lower bound from 7334737807253073920 to 7334737807026794496. Current Snapshot: MvccSnapshot[applied={T|T < 7334737807253073920}]
I20260929 18:44:32.495003 29326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:32.495195 29326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:32.497722 29326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d00d2db811240699db4807f3b631d20: Sending DeleteTablet for 3 replicas of tablet 6a64b379ba8543c596617596a6fdf3a0
I20260929 18:44:32.498600 29712 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:32 UTC) from {username='slave'} at 127.0.0.1:51170
I20260929 18:44:32.498615 29450 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:32 UTC) from {username='slave'} at 127.0.0.1:34956
I20260929 18:44:32.498811 29581 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a64b379ba8543c596617596a6fdf3a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:32 UTC) from {username='slave'} at 127.0.0.1:44294
I20260929 18:44:32.499198 29818 tablet_replica.cc:330] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f: stopping tablet replica
I20260929 18:44:32.499397 29819 tablet_replica.cc:330] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: stopping tablet replica
I20260929 18:44:32.499567 29820 tablet_replica.cc:330] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: stopping tablet replica
I20260929 18:44:32.499547 29818 raft_consensus.cc:2229] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:32.499799 29819 raft_consensus.cc:2229] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:44:32.499840 29820 raft_consensus.cc:2229] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:32.499886 29818 raft_consensus.cc:2258] T 6a64b379ba8543c596617596a6fdf3a0 P 79edaf74477d4b43a71f5e1b9a26d92f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:32.499989 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29388
I20260929 18:44:32.500116 29820 raft_consensus.cc:2258] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:32.500418 29819 raft_consensus.cc:2258] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:32.501952 29820 ts_tablet_manager.cc:1905] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:32.502483 29819 ts_tablet_manager.cc:1905] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:32.504772 29820 ts_tablet_manager.cc:1918] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:32.504874 29820 log.cc:1196] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/6a64b379ba8543c596617596a6fdf3a0
I20260929 18:44:32.505139 29819 ts_tablet_manager.cc:1918] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:32.505198 29820 ts_tablet_manager.cc:1939] T 6a64b379ba8543c596617596a6fdf3a0 P eb570610b44e4db2b15c90e7a8fc4526: Deleting consensus metadata
I20260929 18:44:32.505244 29819 log.cc:1196] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/6a64b379ba8543c596617596a6fdf3a0
I20260929 18:44:32.505522 29819 ts_tablet_manager.cc:1939] T 6a64b379ba8543c596617596a6fdf3a0 P 2647469dd38546e9aca7b7e4995aa034: Deleting consensus metadata
I20260929 18:44:32.506511 29313 catalog_manager.cc:4841] TS 2647469dd38546e9aca7b7e4995aa034 (127.11.118.195:37581): tablet 6a64b379ba8543c596617596a6fdf3a0 (table test-table [id=9d4beeff11b6454f833e85621775f615]) successfully deleted
I20260929 18:44:32.506511 29315 catalog_manager.cc:4841] TS eb570610b44e4db2b15c90e7a8fc4526 (127.11.118.194:39395): tablet 6a64b379ba8543c596617596a6fdf3a0 (table test-table [id=9d4beeff11b6454f833e85621775f615]) successfully deleted
W20260929 18:44:32.508771 29312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:37715 (error 108)
I20260929 18:44:32.509619 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29519
I20260929 18:44:32.509873 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:32.509545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37715 (local address 127.0.0.1:43784)
0929 18:44:32.509714 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:32.509792 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 18:44:32.510308 29312 catalog_manager.cc:4584] TS 79edaf74477d4b43a71f5e1b9a26d92f (127.11.118.193:37715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a64b379ba8543c596617596a6fdf3a0: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37715: connect: Connection refused (error 111)
I20260929 18:44:32.516738 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29650
I20260929 18:44:32.522526 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29297
2026-09-29T18:44:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1185 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:32 dist-test-slave-cgv7 krb5kdc[29824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:32 dist-test-slave-cgv7 krb5kdc[29824](info): set up 2 sockets
Sep 29 18:44:32 dist-test-slave-cgv7 krb5kdc[29824](info): commencing operation
krb5kdc: starting...
W20260929 18:44:33.578192 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:33 dist-test-slave-cgv7 krb5kdc[29824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:44:33.709980 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42535
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:42535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:33.823719 29837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:33.824075 29837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:33.824159 29837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:33.828436 29837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:33.828540 29837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:33.828575 29837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:33.828614 29837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:33.828651 29837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:33.834393 29837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:42535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:33.835585 29837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:33.836709 29837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:33.845567 29845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:33.845577 29842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:33.845713 29837 server_base.cc:1034] running on GCE node
W20260929 18:44:33.845567 29843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:33.846565 29837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:33.847731 29837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:33.848915 29837 hybrid_clock.cc:648] HybridClock initialized: now 1790707473848897 us; error 32 us; skew 500 ppm
Sep 29 18:44:33 dist-test-slave-cgv7 krb5kdc[29824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:33.851994 29837 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:33.852972 29837 webserver.cc:460] Webserver started at http://127.11.118.254:44795/ using document root <none> and password file <none>
I20260929 18:44:33.853564 29837 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:33.853631 29837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:33.853865 29837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:33.856209 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "a81d27070e7c408da1d42f2d5c7a89d1"
format_stamp: "Formatted at 2026-09-29 18:44:33 on dist-test-slave-cgv7"
server_key: "f7aaf0729305f175b4fd64f339492784"
server_key_iv: "797ed7307c183476d81518514cc4f8e3"
server_key_version: "encryptionkey@0"
I20260929 18:44:33.856846 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "a81d27070e7c408da1d42f2d5c7a89d1"
format_stamp: "Formatted at 2026-09-29 18:44:33 on dist-test-slave-cgv7"
server_key: "f7aaf0729305f175b4fd64f339492784"
server_key_iv: "797ed7307c183476d81518514cc4f8e3"
server_key_version: "encryptionkey@0"
I20260929 18:44:33.861063 29837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:33.863925 29852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:33.865379 29837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:44:33.865548 29837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "a81d27070e7c408da1d42f2d5c7a89d1"
format_stamp: "Formatted at 2026-09-29 18:44:33 on dist-test-slave-cgv7"
server_key: "f7aaf0729305f175b4fd64f339492784"
server_key_iv: "797ed7307c183476d81518514cc4f8e3"
server_key_version: "encryptionkey@0"
I20260929 18:44:33.865684 29837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:33.888552 29837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:33.892146 29837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:33.892390 29837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:33.899427 29837 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:42535
I20260929 18:44:33.899446 29904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:42535 every 8 connection(s)
I20260929 18:44:33.900449 29837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:33.903252 29905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:33.906905 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29837
I20260929 18:44:33.907056 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:33.907385 11739 external_mini_cluster.cc:1428] Setting key dd80da58b92fdb5f9ed74ed913630dae
I20260929 18:44:33.909610 29905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: Bootstrap starting.
I20260929 18:44:33.911909 29905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:33.912669 29905 log.cc:824] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:33.914685 29905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: No bootstrap required, opened a new log
Sep 29 18:44:33 dist-test-slave-cgv7 krb5kdc[29824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:33.918918 29905 raft_consensus.cc:348] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } }
I20260929 18:44:33.919183 29905 raft_consensus.cc:374] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:33.919265 29905 raft_consensus.cc:729] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a81d27070e7c408da1d42f2d5c7a89d1, State: Initialized, Role: FOLLOWER
I20260929 18:44:33.919818 29905 consensus_queue.cc:260] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } }
I20260929 18:44:33.919991 29905 raft_consensus.cc:388] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:33.920069 29905 raft_consensus.cc:482] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:33.920182 29905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:33.921329 29905 raft_consensus.cc:504] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } }
I20260929 18:44:33.921679 29905 leader_election.cc:302] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a81d27070e7c408da1d42f2d5c7a89d1; no voters:
I20260929 18:44:33.922019 29905 leader_election.cc:288] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:33.922130 29910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:33.922428 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:33.908989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.11.118.254:42535)
0929 18:44:33.909450 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:33.909465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:33.909502 (+ 37us) server_negotiation.cc:407] Connection header received
0929 18:44:33.910223 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:33.910254 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:33.910679 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:33.911025 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:33.911838 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:33.912964 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:33.913828 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:33.914380 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:33.914557 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:33.917336 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:33.917341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:33.917355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:33.917383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:33.919165 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:33.919759 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:33.919764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:33.919769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:33.919884 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:33.920243 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:33.920245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:33.920246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:33.920681 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:33.920837 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:33.921143 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 18:44:33.921450 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":128,"threads_started":1}
I20260929 18:44:33.923303 29905 sys_catalog.cc:564] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:33.922415 29910 raft_consensus.cc:686] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [term 1 LEADER]: Becoming Leader. State: Replica: a81d27070e7c408da1d42f2d5c7a89d1, State: Running, Role: LEADER
I20260929 18:44:33.923836 29910 consensus_queue.cc:237] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } }
I20260929 18:44:33.925652 29911 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a81d27070e7c408da1d42f2d5c7a89d1. Latest consensus state: current_term: 1 leader_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } } }
I20260929 18:44:33.925799 29911 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:33.925863 29912 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81d27070e7c408da1d42f2d5c7a89d1" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42535 } } }
I20260929 18:44:33.925953 29912 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1 [sys.catalog]: This master's current role is: LEADER
W20260929 18:44:33.928418 29924 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:44:33.928505 29924 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:44:33.928633 29925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:33.929720 29925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:33.935388 29925 catalog_manager.cc:1347] Generated new cluster ID: 620d7e0de0204b4194471d2e50022d54
I20260929 18:44:33.935477 29925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:33.955899 29925 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:33.957249 29925 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:33.962049 29925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: Generated new TSK 0
I20260929 18:44:33.962947 29925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:44:34.001643 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:34.119007 29931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:34.119315 29931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:34.119362 29931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:34.123714 29931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:34.123786 29931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:34.123889 29931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:34.129961 29931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:34.131230 29931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:34.132318 29931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:34.142452 29937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:34.142475 29939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:34.142475 29936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:34.142804 29931 server_base.cc:1034] running on GCE node
I20260929 18:44:34.143261 29931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:34.144011 29931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:34.145200 29931 hybrid_clock.cc:648] HybridClock initialized: now 1790707474145184 us; error 31 us; skew 500 ppm
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:34.154017 29931 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:44:34.155218 29931 webserver.cc:460] Webserver started at http://127.11.118.193:37201/ using document root <none> and password file <none>
I20260929 18:44:34.155860 29931 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:34.155929 29931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:34.156162 29931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:34.158460 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "4d66253da9e14cf6bf14179a7f3dd72c"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "9edc44b4909e67ad1922f7782faea8ef"
server_key_iv: "ebc5d7369304d013bca896c6c4c2e48b"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.159080 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "4d66253da9e14cf6bf14179a7f3dd72c"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "9edc44b4909e67ad1922f7782faea8ef"
server_key_iv: "ebc5d7369304d013bca896c6c4c2e48b"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.163156 29931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:44:34.165774 29946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.167258 29931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:34.167407 29931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "4d66253da9e14cf6bf14179a7f3dd72c"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "9edc44b4909e67ad1922f7782faea8ef"
server_key_iv: "ebc5d7369304d013bca896c6c4c2e48b"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.167533 29931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:34.185192 29931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:34.188581 29931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:34.188827 29931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:34.189457 29931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:34.190400 29931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:34.190457 29931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.190526 29931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:34.190559 29931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.199867 29931 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:38079
I20260929 18:44:34.199889 30059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:38079 every 8 connection(s)
I20260929 18:44:34.200807 29931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:34.209419 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 29931
I20260929 18:44:34.209581 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:34.209933 11739 external_mini_cluster.cc:1428] Setting key b4f66e9ebab44d873308dd52058482c5
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:34.216018 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.202849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:36809 (local address 127.11.118.254:42535)
0929 18:44:34.203000 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.203004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.203764 (+ 760us) server_negotiation.cc:407] Connection header received
0929 18:44:34.205174 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.205177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.205236 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.205320 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:34.206620 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.207182 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.208429 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.208678 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.208796 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.211653 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:34.211656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:34.211658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:34.211683 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:34.213489 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.214042 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.214047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.214051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.214163 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.214579 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.214585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.214589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.214865 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:34.214978 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.215587 (+ 609us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.215758 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 18:44:34.216864 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.203133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42535 (local address 127.11.118.193:36809)
0929 18:44:34.203599 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.203644 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.204945 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.205525 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.205532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.205916 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:34.206452 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.206467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.207326 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.207340 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.208282 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.208292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.208846 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.208850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.209031 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.209759 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:34.209777 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:34.211476 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:34.213626 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.213631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.213645 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.213929 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.214257 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.214260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.214261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.214477 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.215047 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:34.215052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:34.215309 (+ 257us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.215474 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.215927 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":110,"threads_started":1}
I20260929 18:44:34.218214 30060 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42535
I20260929 18:44:34.218470 30060 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:34.218986 30060 heartbeater.cc:511] Master 127.11.118.254:42535 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:44:34.220609 29869 ts_manager.cc:194] Registered new tserver with Master: 4d66253da9e14cf6bf14179a7f3dd72c (127.11.118.193:38079)
I20260929 18:44:34.221756 29869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:36809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:44:34.242000 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:34.358441 30065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:34.358749 30065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:34.358796 30065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:34.363222 30065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:34.363312 30065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:34.363379 30065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:34.369378 30065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:34.370581 30065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:34.371709 30065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:34.381861 30070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:34.381883 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:34.381978 30065 server_base.cc:1034] running on GCE node
W20260929 18:44:34.381913 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:34.382650 30065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:34.383338 30065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:34.384541 30065 hybrid_clock.cc:648] HybridClock initialized: now 1790707474384520 us; error 33 us; skew 500 ppm
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:34.387677 30065 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:44:34.388600 30065 webserver.cc:460] Webserver started at http://127.11.118.194:38593/ using document root <none> and password file <none>
I20260929 18:44:34.389163 30065 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:34.389223 30065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:34.389397 30065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:34.391573 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "5820989eb6f4fa5df8f78644195ad390"
server_key_iv: "192764979a0a38ea39243d74b2992625"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.392105 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "5820989eb6f4fa5df8f78644195ad390"
server_key_iv: "192764979a0a38ea39243d74b2992625"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.396162 30065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 18:44:34.398901 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.400329 30065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:34.400480 30065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "5820989eb6f4fa5df8f78644195ad390"
server_key_iv: "192764979a0a38ea39243d74b2992625"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.400605 30065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:34.420365 30065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:34.424189 30065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:34.424466 30065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:34.425105 30065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:34.426035 30065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:34.426107 30065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.426174 30065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:34.426229 30065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.435559 30065 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:34617
I20260929 18:44:34.435575 30193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:34617 every 8 connection(s)
I20260929 18:44:34.436810 30065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:34.439287 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30065
I20260929 18:44:34.439417 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:34.439744 11739 external_mini_cluster.cc:1428] Setting key 720ab2b49cded077d2ddac6e3370f9ba
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
I20260929 18:44:34.452687 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.438785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:53697 (local address 127.11.118.254:42535)
0929 18:44:34.438949 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.438953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.439991 (+ 1038us) server_negotiation.cc:407] Connection header received
0929 18:44:34.441536 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.441539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.441594 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.441672 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:34.443321 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.443886 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.445202 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.445639 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.445765 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.448782 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:34.448786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:34.448788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:34.448813 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:34.450427 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.450984 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.450988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.450990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.451079 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.451470 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.451475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.451477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.451702 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:34.451821 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.452367 (+ 546us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.452520 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:44:34.453598 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.439205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42535 (local address 127.11.118.194:53697)
0929 18:44:34.439777 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.439866 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.441322 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.441855 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.441862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.442224 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:34.443077 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.443104 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.444043 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.444046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.445060 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.445070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.445795 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.445798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.446045 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.446756 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:34.446784 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:34.448584 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:34.450557 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.450564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.450583 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.450866 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.451184 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.451188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.451190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.451365 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.451818 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:34.451826 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:34.452110 (+ 284us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.452327 (+ 217us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.452689 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":155,"threads_started":1}
I20260929 18:44:34.455070 30194 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42535
I20260929 18:44:34.455489 30194 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:34.456161 30194 heartbeater.cc:511] Master 127.11.118.254:42535 requested a full tablet report, sending...
I20260929 18:44:34.457540 29869 ts_manager.cc:194] Registered new tserver with Master: c88916c6fb3b4fd59a75fe0cbc946a0b (127.11.118.194:34617)
I20260929 18:44:34.458249 29869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:53697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:44:34.470453 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:34.585062 30199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:34.585343 30199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:34.585387 30199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:34.589455 30199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:34.589520 30199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:34.589581 30199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:34.595005 30199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:34.596117 30199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:34.597148 30199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:34.607487 30205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:34.607538 30204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:34.607515 30199 server_base.cc:1034] running on GCE node
W20260929 18:44:34.607533 30207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:34.608078 30199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:34.608824 30199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:34.610060 30199 hybrid_clock.cc:648] HybridClock initialized: now 1790707474610044 us; error 27 us; skew 500 ppm
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:34.613162 30199 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:44:34.614073 30199 webserver.cc:460] Webserver started at http://127.11.118.195:45655/ using document root <none> and password file <none>
I20260929 18:44:34.614696 30199 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:34.614759 30199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:34.614930 30199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:34.617029 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "7d41759987434b55a280d68ff422b86f"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "6824ab6743bd58704d157e85011bd485"
server_key_iv: "03cfe6bc2e7bcfe89e538a45e3dbcbe6"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.617555 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "7d41759987434b55a280d68ff422b86f"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "6824ab6743bd58704d157e85011bd485"
server_key_iv: "03cfe6bc2e7bcfe89e538a45e3dbcbe6"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.621279 30199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:34.624080 30214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.625489 30199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 18:44:34.625587 30199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "7d41759987434b55a280d68ff422b86f"
format_stamp: "Formatted at 2026-09-29 18:44:34 on dist-test-slave-cgv7"
server_key: "6824ab6743bd58704d157e85011bd485"
server_key_iv: "03cfe6bc2e7bcfe89e538a45e3dbcbe6"
server_key_version: "encryptionkey@0"
I20260929 18:44:34.625679 30199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:34.644048 30199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:34.647655 30199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:34.647830 30199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:34.648373 30199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:34.649166 30199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:34.649219 30199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.649261 30199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:34.649276 30199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:34.659135 30199 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:40903
I20260929 18:44:34.659157 30327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:40903 every 8 connection(s)
I20260929 18:44:34.660090 30199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:34.668051 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30199
I20260929 18:44:34.668219 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:34.668627 11739 external_mini_cluster.cc:1428] Setting key 420e814d6997725a673f54af2b31feaf
Sep 29 18:44:34 dist-test-slave-cgv7 krb5kdc[29824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707474, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:34.675480 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.662198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:51387 (local address 127.11.118.254:42535)
0929 18:44:34.662417 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.662422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.663276 (+ 854us) server_negotiation.cc:407] Connection header received
0929 18:44:34.664590 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.664593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.664644 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.664736 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:34.665979 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.666618 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.667944 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.668250 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.668370 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.671661 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:34.671664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:34.671666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:34.671692 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:34.672967 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.673499 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.673504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.673505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.673589 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.673951 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.673954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.673956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.674251 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:34.674445 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.675157 (+ 712us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.675309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260929 18:44:34.676488 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.662619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42535 (local address 127.11.118.195:51387)
0929 18:44:34.663099 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.663149 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.664364 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.664930 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.664937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.665307 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:34.665799 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.665823 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.666743 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.666745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.667747 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.667760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.668414 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.668419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.668630 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.669413 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:34.669438 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:34.671458 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:34.673090 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.673096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.673115 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.673391 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.673690 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:34.673693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:34.673695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:34.673843 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:34.674442 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:34.674449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:34.674723 (+ 274us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.675115 (+ 392us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.675488 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":107,"threads_started":1}
I20260929 18:44:34.677691 30328 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42535
I20260929 18:44:34.678011 30328 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:34.678578 30328 heartbeater.cc:511] Master 127.11.118.254:42535 requested a full tablet report, sending...
I20260929 18:44:34.679677 29869 ts_manager.cc:194] Registered new tserver with Master: 7d41759987434b55a280d68ff422b86f (127.11.118.195:40903)
I20260929 18:44:34.680212 29869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:51387
I20260929 18:44:34.683044 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:34.693421 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.685026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.11.118.254:42535)
0929 18:44:34.685226 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.685231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.685404 (+ 173us) server_negotiation.cc:407] Connection header received
0929 18:44:34.685564 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.685568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.685616 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.685710 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:34.686501 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.687030 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.687878 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.688138 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.688256 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.690001 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:34.690006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:34.690013 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:34.690048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:34.691522 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.692065 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.692070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.692077 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.692149 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:34.692484 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:34.692487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:34.692489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:34.692702 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:34.692863 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.693151 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.693255 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:44:34.696542 29869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:34.699688 29869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:34.714946 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.709020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:34617 (local address 127.0.0.1:49712)
0929 18:44:34.710004 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.710018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.710165 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.710651 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.710658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.710678 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:34.710821 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.710828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.712584 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.712587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.713479 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.713488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714381 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.714484 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714519 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.714658 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.714789 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":104,"threads_started":1}
I20260929 18:44:34.715085 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.708632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:40903 (local address 127.0.0.1:43168)
0929 18:44:34.709904 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.709955 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.710112 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.710828 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.710831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.710848 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:34.711092 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.711098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.712091 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.712097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.713530 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.713541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714666 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.714747 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714766 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.714840 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.714896 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":75,"thread_start_us":91,"threads_started":1}
I20260929 18:44:34.715266 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.708713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:38079 (local address 127.0.0.1:59980)
0929 18:44:34.709995 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.710010 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.710112 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.710839 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.710842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.710853 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:34.711093 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.711098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.712423 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.712427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.713556 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.713563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714851 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.714945 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714967 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.715037 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.715113 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":110,"thread_start_us":79,"threads_started":1}
I20260929 18:44:34.715577 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.709308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.11.118.194:34617)
0929 18:44:34.709723 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.709740 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.710095 (+ 355us) server_negotiation.cc:407] Connection header received
0929 18:44:34.710340 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.710345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.710505 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.710664 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:34.710943 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.712099 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.713612 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714203 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714404 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714544 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.715325 (+ 781us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.715439 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":112,"threads_started":1}
I20260929 18:44:34.715675 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.709479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.11.118.193:38079)
0929 18:44:34.709874 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.709889 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.710251 (+ 362us) server_negotiation.cc:407] Connection header received
0929 18:44:34.710354 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.710359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.710547 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.710690 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:34.711219 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.712295 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.713678 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714545 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714665 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714781 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.715446 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.715541 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":93,"threads_started":1}
I20260929 18:44:34.716620 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.708757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.11.118.195:40903)
0929 18:44:34.709936 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.709950 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.710250 (+ 300us) server_negotiation.cc:407] Connection header received
0929 18:44:34.710357 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.710361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.710547 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.710690 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:34.711233 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.711953 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.713678 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.714546 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.714666 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.714781 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.714892 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.715001 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1082,"thread_start_us":170,"threads_started":1}
I20260929 18:44:34.718438 29994 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 (DEFAULT_TABLE table=test-table [id=1d25ff4ff534472b80ddbecc9a5af82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:34.718479 30128 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 (DEFAULT_TABLE table=test-table [id=1d25ff4ff534472b80ddbecc9a5af82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:34.719102 30262 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 (DEFAULT_TABLE table=test-table [id=1d25ff4ff534472b80ddbecc9a5af82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:34.719481 30128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7042f6dea64871853fa207b5ce8e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:34.719482 29994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7042f6dea64871853fa207b5ce8e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:34.719961 30262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7042f6dea64871853fa207b5ce8e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:34.725785 30345 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: Bootstrap starting.
I20260929 18:44:34.726541 30346 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Bootstrap starting.
I20260929 18:44:34.728359 30347 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Bootstrap starting.
I20260929 18:44:34.728559 30345 tablet_bootstrap.cc:654] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:34.729621 30345 log.cc:824] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:34.730934 30347 tablet_bootstrap.cc:654] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:34.731766 30346 tablet_bootstrap.cc:654] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:34.732456 30347 log.cc:824] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:34.733141 30346 log.cc:824] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:34.734879 30347 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: No bootstrap required, opened a new log
I20260929 18:44:34.735113 30347 ts_tablet_manager.cc:1397] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 18:44:34.735517 30345 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: No bootstrap required, opened a new log
I20260929 18:44:34.735709 30346 tablet_bootstrap.cc:492] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: No bootstrap required, opened a new log
I20260929 18:44:34.735723 30345 ts_tablet_manager.cc:1397] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 18:44:34.735955 30346 ts_tablet_manager.cc:1397] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 18:44:34.739378 30347 raft_consensus.cc:348] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.739718 30347 raft_consensus.cc:374] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:34.739778 30347 raft_consensus.cc:729] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d41759987434b55a280d68ff422b86f, State: Initialized, Role: FOLLOWER
I20260929 18:44:34.739763 30345 raft_consensus.cc:348] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.740031 30345 raft_consensus.cc:374] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:34.740087 30345 raft_consensus.cc:729] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d66253da9e14cf6bf14179a7f3dd72c, State: Initialized, Role: FOLLOWER
I20260929 18:44:34.740249 30346 raft_consensus.cc:348] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.740449 30347 consensus_queue.cc:260] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.740530 30346 raft_consensus.cc:374] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:34.740592 30346 raft_consensus.cc:729] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88916c6fb3b4fd59a75fe0cbc946a0b, State: Initialized, Role: FOLLOWER
I20260929 18:44:34.740550 30345 consensus_queue.cc:260] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.741036 30346 consensus_queue.cc:260] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.741266 30060 heartbeater.cc:503] Master 127.11.118.254:42535 was elected leader, sending a full tablet report...
I20260929 18:44:34.741653 30345 ts_tablet_manager.cc:1428] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:34.741964 30194 heartbeater.cc:503] Master 127.11.118.254:42535 was elected leader, sending a full tablet report...
I20260929 18:44:34.742074 30346 ts_tablet_manager.cc:1428] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:34.742672 30347 ts_tablet_manager.cc:1428] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 18:44:34.743050 30328 heartbeater.cc:503] Master 127.11.118.254:42535 was elected leader, sending a full tablet report...
I20260929 18:44:34.838018 30351 raft_consensus.cc:482] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:34.838222 30351 raft_consensus.cc:504] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.839430 30351 leader_election.cc:288] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d66253da9e14cf6bf14179a7f3dd72c (127.11.118.193:38079), c88916c6fb3b4fd59a75fe0cbc946a0b (127.11.118.194:34617)
I20260929 18:44:34.843204 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.839690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:34617 (local address 127.11.118.195:52943)
0929 18:44:34.839937 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.839954 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.840064 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.840348 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.840351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.840372 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:34.840509 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.840515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.841239 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.841243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.842045 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.842054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.842766 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.842885 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.842917 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.842998 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.843071 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260929 18:44:34.843231 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.839819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:52943 (local address 127.11.118.194:34617)
0929 18:44:34.839942 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.839949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.840026 (+ 77us) server_negotiation.cc:407] Connection header received
0929 18:44:34.840178 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.840181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.840226 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.840325 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:34.840628 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.841124 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842181 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.842643 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842776 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.842831 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.843019 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.843110 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:44:34.843282 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.839804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:48555 (local address 127.11.118.193:38079)
0929 18:44:34.839950 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.839956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.840260 (+ 304us) server_negotiation.cc:407] Connection header received
0929 18:44:34.840402 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.840405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.840457 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.840542 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:34.840813 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.841298 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842332 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.842730 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842879 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.842931 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.843104 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.843176 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:44:34.843668 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.839694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:38079 (local address 127.11.118.195:48555)
0929 18:44:34.840148 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.840163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:34.840282 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:34.840555 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:34.840558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:34.840573 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:34.840698 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.840703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.841411 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.841418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.842182 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:34.842190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.842857 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.842860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:34.842963 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.842991 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 18:44:34.843453 (+ 462us) client_negotiation.cc:240] Negotiation successful
0929 18:44:34.843544 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":32,"thread_start_us":172,"threads_started":1}
I20260929 18:44:34.844017 30148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7042f6dea64871853fa207b5ce8e42" candidate_uuid: "7d41759987434b55a280d68ff422b86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" is_pre_election: true
I20260929 18:44:34.844319 30014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7042f6dea64871853fa207b5ce8e42" candidate_uuid: "7d41759987434b55a280d68ff422b86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" is_pre_election: true
I20260929 18:44:34.844384 30148 raft_consensus.cc:2454] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d41759987434b55a280d68ff422b86f in term 0.
I20260929 18:44:34.844599 30014 raft_consensus.cc:2454] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d41759987434b55a280d68ff422b86f in term 0.
I20260929 18:44:34.844859 30216 leader_election.cc:302] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d41759987434b55a280d68ff422b86f, c88916c6fb3b4fd59a75fe0cbc946a0b; no voters:
I20260929 18:44:34.845139 30351 raft_consensus.cc:2781] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:34.845208 30351 raft_consensus.cc:482] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:34.845237 30351 raft_consensus.cc:3037] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:34.846246 30351 raft_consensus.cc:504] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.846644 30351 leader_election.cc:288] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [CANDIDATE]: Term 1 election: Requested vote from peers 4d66253da9e14cf6bf14179a7f3dd72c (127.11.118.193:38079), c88916c6fb3b4fd59a75fe0cbc946a0b (127.11.118.194:34617)
I20260929 18:44:34.847021 30148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7042f6dea64871853fa207b5ce8e42" candidate_uuid: "7d41759987434b55a280d68ff422b86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b"
I20260929 18:44:34.847065 30014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7042f6dea64871853fa207b5ce8e42" candidate_uuid: "7d41759987434b55a280d68ff422b86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d66253da9e14cf6bf14179a7f3dd72c"
I20260929 18:44:34.847152 30148 raft_consensus.cc:3037] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:34.847172 30014 raft_consensus.cc:3037] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:34.848451 30148 raft_consensus.cc:2454] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d41759987434b55a280d68ff422b86f in term 1.
I20260929 18:44:34.848451 30014 raft_consensus.cc:2454] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d41759987434b55a280d68ff422b86f in term 1.
I20260929 18:44:34.848750 30216 leader_election.cc:302] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d41759987434b55a280d68ff422b86f, c88916c6fb3b4fd59a75fe0cbc946a0b; no voters:
I20260929 18:44:34.848976 30351 raft_consensus.cc:2781] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:34.849205 30351 raft_consensus.cc:686] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 1 LEADER]: Becoming Leader. State: Replica: 7d41759987434b55a280d68ff422b86f, State: Running, Role: LEADER
I20260929 18:44:34.849535 30351 consensus_queue.cc:237] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } }
I20260929 18:44:34.852849 29867 catalog_manager.cc:5495] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d41759987434b55a280d68ff422b86f (127.11.118.195). New cstate: current_term: 1 leader_uuid: "7d41759987434b55a280d68ff422b86f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d41759987434b55a280d68ff422b86f" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 } health_report { overall_health: UNKNOWN } } }
I20260929 18:44:34.869197 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.865559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.11.118.195:40903)
0929 18:44:34.865694 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:34.865698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:34.865712 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:44:34.865863 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:34.865866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:34.865924 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:34.866037 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:34.866308 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.866882 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.867722 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:34.867948 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:34.868058 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:34.868354 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:34.868743 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:34.868911 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:34.868973 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 18:44:34.869009 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:44:34.875313 30148 raft_consensus.cc:1261] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 1 FOLLOWER]: Refusing update from remote peer 7d41759987434b55a280d68ff422b86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:34.875468 30014 raft_consensus.cc:1261] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 1 FOLLOWER]: Refusing update from remote peer 7d41759987434b55a280d68ff422b86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:34.876168 30355 consensus_queue.cc:1035] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c88916c6fb3b4fd59a75fe0cbc946a0b" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:34.876483 30351 consensus_queue.cc:1035] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d66253da9e14cf6bf14179a7f3dd72c" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 38079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:34.883849 30359 mvcc.cc:205] Tried to move back new op lower bound from 7334737817081438208 to 7334737816986800128. Current Snapshot: MvccSnapshot[applied={T|T < 7334737817081438208}]
I20260929 18:44:34.892306 29868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:34.892463 29868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:34.894914 29868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a81d27070e7c408da1d42f2d5c7a89d1: Sending DeleteTablet for 3 replicas of tablet 5b7042f6dea64871853fa207b5ce8e42
I20260929 18:44:34.895660 30262 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:34 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:43168
I20260929 18:44:34.895704 29994 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:34 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:59980
I20260929 18:44:34.895900 30128 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7042f6dea64871853fa207b5ce8e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:34 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:49712
I20260929 18:44:34.896190 30368 tablet_replica.cc:330] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c: stopping tablet replica
I20260929 18:44:34.896191 30367 tablet_replica.cc:330] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: stopping tablet replica
I20260929 18:44:34.896483 30368 raft_consensus.cc:2229] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:34.896483 30367 raft_consensus.cc:2229] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:44:34.896787 30368 raft_consensus.cc:2258] T 5b7042f6dea64871853fa207b5ce8e42 P 4d66253da9e14cf6bf14179a7f3dd72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:34.896857 30367 raft_consensus.cc:2258] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:34.897185 30369 tablet_replica.cc:330] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: stopping tablet replica
I20260929 18:44:34.897286 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29931
I20260929 18:44:34.897470 30369 raft_consensus.cc:2229] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:34.897768 30369 raft_consensus.cc:2258] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:34.899398 30367 ts_tablet_manager.cc:1905] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:34.899982 30369 ts_tablet_manager.cc:1905] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:34.902211 30367 ts_tablet_manager.cc:1918] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:34.902359 30367 log.cc:1196] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/5b7042f6dea64871853fa207b5ce8e42
I20260929 18:44:34.902720 30367 ts_tablet_manager.cc:1939] T 5b7042f6dea64871853fa207b5ce8e42 P 7d41759987434b55a280d68ff422b86f: Deleting consensus metadata
I20260929 18:44:34.902706 30369 ts_tablet_manager.cc:1918] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:34.902796 30369 log.cc:1196] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/5b7042f6dea64871853fa207b5ce8e42
I20260929 18:44:34.903126 30369 ts_tablet_manager.cc:1939] T 5b7042f6dea64871853fa207b5ce8e42 P c88916c6fb3b4fd59a75fe0cbc946a0b: Deleting consensus metadata
I20260929 18:44:34.904145 29856 catalog_manager.cc:4841] TS 7d41759987434b55a280d68ff422b86f (127.11.118.195:40903): tablet 5b7042f6dea64871853fa207b5ce8e42 (table test-table [id=1d25ff4ff534472b80ddbecc9a5af82a]) successfully deleted
I20260929 18:44:34.904363 29854 catalog_manager.cc:4841] TS c88916c6fb3b4fd59a75fe0cbc946a0b (127.11.118.194:34617): tablet 5b7042f6dea64871853fa207b5ce8e42 (table test-table [id=1d25ff4ff534472b80ddbecc9a5af82a]) successfully deleted
W20260929 18:44:34.908862 29855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:38079 (error 108)
I20260929 18:44:34.909615 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30065
I20260929 18:44:34.910458 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:34.909749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:38079 (local address 127.0.0.1:59988)
0929 18:44:34.910308 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:34.910376 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445}
W20260929 18:44:34.910831 29855 catalog_manager.cc:4584] TS 4d66253da9e14cf6bf14179a7f3dd72c (127.11.118.193:38079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b7042f6dea64871853fa207b5ce8e42: Network error: Client connection negotiation failed: client connection to 127.11.118.193:38079: connect: Connection refused (error 111)
I20260929 18:44:34.917003 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30199
I20260929 18:44:34.923269 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 29837
2026-09-29T18:44:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2401 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T18:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:34Z Disabled control of system clock
I20260929 18:44:34.953998 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42199
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:42199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:35.067260 30374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:35.067591 30374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:35.067629 30374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:35.072000 30374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:35.072074 30374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:35.072091 30374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:35.072109 30374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:35.072125 30374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:35.077646 30374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:42199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:42199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:35.078994 30374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:35.080426 30374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:44:35.089830 30374 server_base.cc:1034] running on GCE node
W20260929 18:44:35.089764 30379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.089752 30380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.089752 30382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.090791 30374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:35.091967 30374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:35.093163 30374 hybrid_clock.cc:648] HybridClock initialized: now 1790707475093145 us; error 34 us; skew 500 ppm
I20260929 18:44:35.095218 30374 webserver.cc:460] Webserver started at http://127.11.118.254:33367/ using document root <none> and password file <none>
I20260929 18:44:35.095829 30374 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:35.095935 30374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:35.096174 30374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:35.098261 30374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "4985fbee2c274af7856889dfd61dd12d"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "668b349c7d2ceef6aa96598ae75d3e79"
server_key_iv: "bcf3d464c02a209a9e14fabf3219d287"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.098904 30374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "4985fbee2c274af7856889dfd61dd12d"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "668b349c7d2ceef6aa96598ae75d3e79"
server_key_iv: "bcf3d464c02a209a9e14fabf3219d287"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.102731 30374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:44:35.105334 30388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.106709 30374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:35.106860 30374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "4985fbee2c274af7856889dfd61dd12d"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "668b349c7d2ceef6aa96598ae75d3e79"
server_key_iv: "bcf3d464c02a209a9e14fabf3219d287"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.106983 30374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:35.125204 30374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:35.125963 30374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:35.126183 30374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:35.133368 30374 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:42199
I20260929 18:44:35.133387 30440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:42199 every 8 connection(s)
I20260929 18:44:35.134379 30374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:35.137774 30441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:35.140476 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30374
I20260929 18:44:35.140585 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:35.140898 11739 external_mini_cluster.cc:1428] Setting key 4ca11eb65706c4dc80bc73a0cd771453
I20260929 18:44:35.144109 30441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: Bootstrap starting.
I20260929 18:44:35.146752 30441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:35.147576 30441 log.cc:824] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:35.150074 30441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: No bootstrap required, opened a new log
I20260929 18:44:35.150782 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.142426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.11.118.254:42199)
0929 18:44:35.143035 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.143051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.143087 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:44:35.143698 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.143715 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.144116 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.144435 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:35.144791 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.145738 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.146965 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.147451 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.147578 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.148472 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:35.148478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:35.148496 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:35.149058 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:35.149130 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:35.149157 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:35.149261 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.149446 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.149741 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":128,"threads_started":1}
I20260929 18:44:35.154211 30441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } }
I20260929 18:44:35.154508 30441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:35.154600 30441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985fbee2c274af7856889dfd61dd12d, State: Initialized, Role: FOLLOWER
I20260929 18:44:35.155052 30441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } }
I20260929 18:44:35.155179 30441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:35.155218 30441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:35.155289 30441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:35.156191 30441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } }
I20260929 18:44:35.156491 30441 leader_election.cc:302] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4985fbee2c274af7856889dfd61dd12d; no voters:
I20260929 18:44:35.156747 30441 leader_election.cc:288] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:35.156962 30446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:35.157203 30446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [term 1 LEADER]: Becoming Leader. State: Replica: 4985fbee2c274af7856889dfd61dd12d, State: Running, Role: LEADER
I20260929 18:44:35.157486 30446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } }
I20260929 18:44:35.158039 30441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:35.159242 30447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4985fbee2c274af7856889dfd61dd12d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } } }
I20260929 18:44:35.159243 30448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4985fbee2c274af7856889dfd61dd12d. Latest consensus state: current_term: 1 leader_uuid: "4985fbee2c274af7856889dfd61dd12d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985fbee2c274af7856889dfd61dd12d" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 42199 } } }
I20260929 18:44:35.159409 30447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:35.159409 30448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:35.159782 30455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:35.162992 30455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:35.168651 30455 catalog_manager.cc:1347] Generated new cluster ID: cf63e5984ae94770976cfe2d146c14f1
I20260929 18:44:35.168725 30455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:35.190465 30455 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:35.191776 30455 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:35.199460 30455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: Generated new TSK 0
I20260929 18:44:35.200229 30455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:35.208804 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--builtin_ntp_servers=127.11.118.212:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:35.327680 30465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:35.327975 30465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:35.328015 30465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:35.332327 30465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:35.332402 30465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:35.332463 30465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:35.337426 30465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:35.338675 30465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:35.340201 30465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:35.350518 30470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.350670 30465 server_base.cc:1034] running on GCE node
W20260929 18:44:35.350518 30471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.350518 30473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.351716 30465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:35.352504 30465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:35.353717 30465 hybrid_clock.cc:648] HybridClock initialized: now 1790707475353672 us; error 62 us; skew 500 ppm
I20260929 18:44:35.355942 30465 webserver.cc:460] Webserver started at http://127.11.118.193:39153/ using document root <none> and password file <none>
I20260929 18:44:35.356564 30465 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:35.356663 30465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:35.356920 30465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:35.359297 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "c5ed945a4ecf4e4397f5952645cad184"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "522dd1b2de27855b1b7e25c41f1ba000"
server_key_iv: "e08d60abfbd3841580fc95145d63bce8"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.359967 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "c5ed945a4ecf4e4397f5952645cad184"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "522dd1b2de27855b1b7e25c41f1ba000"
server_key_iv: "e08d60abfbd3841580fc95145d63bce8"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.363795 30465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:44:35.366384 30479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.367645 30465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:44:35.367785 30465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "c5ed945a4ecf4e4397f5952645cad184"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "522dd1b2de27855b1b7e25c41f1ba000"
server_key_iv: "e08d60abfbd3841580fc95145d63bce8"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.367920 30465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:35.385176 30465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:35.385942 30465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:35.386147 30465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:35.386763 30465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:35.387719 30465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:35.387789 30465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.387862 30465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:35.387917 30465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.396914 30465 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:36583
I20260929 18:44:35.396932 30592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:36583 every 8 connection(s)
I20260929 18:44:35.397871 30465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:35.406564 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30465
I20260929 18:44:35.406740 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:35.407158 11739 external_mini_cluster.cc:1428] Setting key 7807fb98f40daf7131540fee35318a2a
I20260929 18:44:35.407575 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.399842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:59247 (local address 127.11.118.254:42199)
0929 18:44:35.400002 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.400007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.401110 (+ 1103us) server_negotiation.cc:407] Connection header received
0929 18:44:35.402519 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.402522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.402569 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.402653 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:35.403587 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.404167 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.405367 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.405577 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.405665 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.406958 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:35.406964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:35.406967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:35.406984 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:35.407009 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:35.407020 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:35.407096 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.407293 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.407382 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 18:44:35.409083 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.400321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42199 (local address 127.11.118.193:59247)
0929 18:44:35.400953 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.400994 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.402306 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.402821 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.402828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.402917 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:35.403437 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.403449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.404283 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.404286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.405254 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.405263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.405712 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.405714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.405861 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.406596 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:35.406598 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:35.406638 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:35.406729 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:35.407112 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:35.407121 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:35.407127 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.407819 (+ 692us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.408152 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260929 18:44:35.409349 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--builtin_ntp_servers=127.11.118.212:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:35.410552 30593 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42199
I20260929 18:44:35.410835 30593 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:35.411453 30593 heartbeater.cc:511] Master 127.11.118.254:42199 requested a full tablet report, sending...
I20260929 18:44:35.413125 30404 ts_manager.cc:194] Registered new tserver with Master: c5ed945a4ecf4e4397f5952645cad184 (127.11.118.193:36583)
I20260929 18:44:35.414966 30404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.193:59247
W20260929 18:44:35.528101 30596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:35.528415 30596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:35.528457 30596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:35.532655 30596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:35.532730 30596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:35.532789 30596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:35.537789 30596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:35.538960 30596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:35.540405 30596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:35.550513 30604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.550511 30602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.550523 30601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.550735 30596 server_base.cc:1034] running on GCE node
I20260929 18:44:35.551367 30596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:35.552098 30596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:35.553318 30596 hybrid_clock.cc:648] HybridClock initialized: now 1790707475553299 us; error 33 us; skew 500 ppm
I20260929 18:44:35.555451 30596 webserver.cc:460] Webserver started at http://127.11.118.194:42851/ using document root <none> and password file <none>
I20260929 18:44:35.556164 30596 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:35.556260 30596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:35.556517 30596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:35.558895 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "e90cea4a94304595874d1d20957a60dd"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "25c1f75552fed3f2ab6a6cbbbd8c9ac3"
server_key_iv: "7487b8537bc4bc4539288b10b75a0600"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.559592 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "e90cea4a94304595874d1d20957a60dd"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "25c1f75552fed3f2ab6a6cbbbd8c9ac3"
server_key_iv: "7487b8537bc4bc4539288b10b75a0600"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.563589 30596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 18:44:35.566214 30610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.567694 30596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:44:35.567853 30596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "e90cea4a94304595874d1d20957a60dd"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "25c1f75552fed3f2ab6a6cbbbd8c9ac3"
server_key_iv: "7487b8537bc4bc4539288b10b75a0600"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.567989 30596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:35.584438 30596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:35.585180 30596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:35.585397 30596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:35.585999 30596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:35.587050 30596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:35.587124 30596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.587194 30596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:35.587241 30596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.596282 30596 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:39357
I20260929 18:44:35.596303 30723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:39357 every 8 connection(s)
I20260929 18:44:35.597326 30596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:35.606658 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.599152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:45405 (local address 127.11.118.254:42199)
0929 18:44:35.599331 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.599336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.600434 (+ 1098us) server_negotiation.cc:407] Connection header received
0929 18:44:35.601776 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.601778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.601833 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.601913 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:35.602928 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.603450 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.604611 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.604981 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.605078 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.606165 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:35.606168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:35.606170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:35.606184 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:35.606203 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:35.606210 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:35.606339 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.606479 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.606536 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 18:44:35.607105 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30596
I20260929 18:44:35.607237 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:35.607537 11739 external_mini_cluster.cc:1428] Setting key 0febdd7f78d4f9d88140469197a6b0e9
I20260929 18:44:35.607872 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.599609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42199 (local address 127.11.118.194:45405)
0929 18:44:35.600271 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.600312 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.601574 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.602111 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.602118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.602211 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:35.602764 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.602779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.603567 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.603569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.604474 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.604482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.605132 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.605134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.605277 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.605889 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:35.605890 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:35.605933 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:35.606000 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:35.606343 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:35.606350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:35.606355 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.606460 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.606739 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
I20260929 18:44:35.609187 30724 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42199
I20260929 18:44:35.609431 30724 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:35.610057 30724 heartbeater.cc:511] Master 127.11.118.254:42199 requested a full tablet report, sending...
I20260929 18:44:35.610189 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--builtin_ntp_servers=127.11.118.212:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:35.611289 30404 ts_manager.cc:194] Registered new tserver with Master: e90cea4a94304595874d1d20957a60dd (127.11.118.194:39357)
I20260929 18:44:35.611864 30404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.194:45405
W20260929 18:44:35.726504 30727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:35.726799 30727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:35.726838 30727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:35.731088 30727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:35.731161 30727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:35.731220 30727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:35.736155 30727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:35.737310 30727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:35.738722 30727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:35.748953 30735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:35.748960 30732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.749161 30727 server_base.cc:1034] running on GCE node
W20260929 18:44:35.748958 30733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:35.749712 30727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:35.750569 30727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:35.751798 30727 hybrid_clock.cc:648] HybridClock initialized: now 1790707475751780 us; error 31 us; skew 500 ppm
I20260929 18:44:35.753930 30727 webserver.cc:460] Webserver started at http://127.11.118.195:40283/ using document root <none> and password file <none>
I20260929 18:44:35.754599 30727 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:35.754669 30727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:35.754863 30727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:35.757022 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "148cc672146c482da3c4789829ce5f67"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "d5decb429c3dd2bb0a3dc89ba030260a"
server_key_iv: "a4d1912e293462afb97dba59bc0f1cb2"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.757556 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "148cc672146c482da3c4789829ce5f67"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "d5decb429c3dd2bb0a3dc89ba030260a"
server_key_iv: "a4d1912e293462afb97dba59bc0f1cb2"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.761682 30727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:35.764267 30741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.765539 30727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:44:35.765710 30727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "148cc672146c482da3c4789829ce5f67"
format_stamp: "Formatted at 2026-09-29 18:44:35 on dist-test-slave-cgv7"
server_key: "d5decb429c3dd2bb0a3dc89ba030260a"
server_key_iv: "a4d1912e293462afb97dba59bc0f1cb2"
server_key_version: "encryptionkey@0"
I20260929 18:44:35.765836 30727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:35.786158 30727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:35.787163 30727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:35.787417 30727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:35.788101 30727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:35.789090 30727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:35.789137 30727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.789177 30727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:35.789234 30727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:35.798194 30727 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:34851
I20260929 18:44:35.798215 30854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:34851 every 8 connection(s)
I20260929 18:44:35.799142 30727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:35.807879 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30727
I20260929 18:44:35.808017 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:35.808384 11739 external_mini_cluster.cc:1428] Setting key fff4e168b617f8912017e2b18a1a0c20
I20260929 18:44:35.808823 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.800946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:41945 (local address 127.11.118.254:42199)
0929 18:44:35.801110 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.801114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.802141 (+ 1027us) server_negotiation.cc:407] Connection header received
0929 18:44:35.803586 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.803589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.803638 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.803725 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:35.804675 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.805223 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.806571 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.806802 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.806898 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.808221 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:35.808224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:35.808228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:35.808246 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:35.808273 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:35.808283 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:35.808371 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.808599 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.808671 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 18:44:35.809908 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.801329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:42199 (local address 127.11.118.195:41945)
0929 18:44:35.801985 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.802027 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.803360 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.803907 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.803914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.804001 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:35.804513 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.804526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.805384 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.805388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.806412 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.806424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.806974 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.806977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.807124 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.807870 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:35.807872 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:35.807917 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:35.807988 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:35.808398 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:35.808406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:35.808411 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.808579 (+ 168us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.808873 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":109,"threads_started":1}
I20260929 18:44:35.811028 30855 heartbeater.cc:347] Connected to a master server at 127.11.118.254:42199
I20260929 18:44:35.811267 30855 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:35.811905 30855 heartbeater.cc:511] Master 127.11.118.254:42199 requested a full tablet report, sending...
I20260929 18:44:35.812974 30404 ts_manager.cc:194] Registered new tserver with Master: 148cc672146c482da3c4789829ce5f67 (127.11.118.195:34851)
I20260929 18:44:35.813556 30404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.195:41945
I20260929 18:44:35.822813 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:35.829068 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.824947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.11.118.254:42199)
0929 18:44:35.825140 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.825151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.825280 (+ 129us) server_negotiation.cc:407] Connection header received
0929 18:44:35.825432 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.825435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.825484 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.825565 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:35.825853 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.826473 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.827349 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.827573 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.827683 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.828576 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:35.828579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:35.828582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:35.828595 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:35.828621 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:35.828629 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:35.828754 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.828881 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.828918 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 18:44:35.832227 30404 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:35.835216 30404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:35.850313 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.844543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36583 (local address 127.0.0.1:40500)
0929 18:44:35.845657 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.845673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.845754 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.846184 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.846187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.846203 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:35.846316 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.846324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847409 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.847412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.848696 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.848708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.849723 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.849727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.849836 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.849883 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.849993 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.850115 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":31,"thread_start_us":84,"threads_started":1}
I20260929 18:44:35.850703 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.844951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.11.118.193:36583)
0929 18:44:35.845738 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.845750 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.845773 (+ 23us) server_negotiation.cc:407] Connection header received
0929 18:44:35.845855 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.845861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.846052 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.846209 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:35.846414 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847257 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.848836 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.849569 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.849695 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.849903 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.850416 (+ 513us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.850534 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":118,"threads_started":1}
I20260929 18:44:35.850970 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.845219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.11.118.195:34851)
0929 18:44:35.845991 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.846007 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.846028 (+ 21us) server_negotiation.cc:407] Connection header received
0929 18:44:35.846097 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.846102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.846355 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.846494 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:35.846641 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847728 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.849161 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.849977 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.850095 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.850254 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.850703 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.850821 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":153,"threads_started":1}
I20260929 18:44:35.850986 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.844325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.11.118.194:39357)
0929 18:44:35.844914 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:35.844928 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:35.844959 (+ 31us) server_negotiation.cc:407] Connection header received
0929 18:44:35.845043 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:35.845048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:35.845256 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:35.845432 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:35.845943 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847212 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.849216 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.849852 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.850017 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.850178 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:35.850231 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 18:44:35.850362 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":103,"threads_started":1}
I20260929 18:44:35.850776 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.844124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:39357 (local address 127.0.0.1:53402)
0929 18:44:35.844662 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.844713 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.844863 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.845428 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.845435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.845463 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:35.845606 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.845615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847338 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.847342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.849084 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.849094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.850008 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.850014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.850119 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.850142 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.850202 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.850318 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":218,"thread_start_us":113,"threads_started":1}
I20260929 18:44:35.852609 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:35.844884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34851 (local address 127.0.0.1:33212)
0929 18:44:35.845794 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:35.845808 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:35.845893 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:35.846469 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:35.846472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:35.846488 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:35.846557 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.846563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.847849 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.847853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.849030 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:35.849041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:35.850338 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:35.850345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:35.850431 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:35.850452 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 18:44:35.850514 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 18:44:35.850572 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":25,"thread_start_us":88,"threads_started":1}
I20260929 18:44:35.854067 30658 tablet_service.cc:1467] Processing CreateTablet for tablet 889f2fbb437447588d482c8b3c98f211 (DEFAULT_TABLE table=test-table [id=18d871ec82884952a49dfbad88686d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:35.854017 30527 tablet_service.cc:1467] Processing CreateTablet for tablet 889f2fbb437447588d482c8b3c98f211 (DEFAULT_TABLE table=test-table [id=18d871ec82884952a49dfbad88686d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:35.855082 30658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889f2fbb437447588d482c8b3c98f211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:35.855120 30527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889f2fbb437447588d482c8b3c98f211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:35.855152 30789 tablet_service.cc:1467] Processing CreateTablet for tablet 889f2fbb437447588d482c8b3c98f211 (DEFAULT_TABLE table=test-table [id=18d871ec82884952a49dfbad88686d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:35.856076 30789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889f2fbb437447588d482c8b3c98f211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:35.862032 30872 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Bootstrap starting.
I20260929 18:44:35.863404 30874 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Bootstrap starting.
I20260929 18:44:35.863667 30873 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Bootstrap starting.
I20260929 18:44:35.864401 30872 tablet_bootstrap.cc:654] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:35.865130 30872 log.cc:824] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:35.865599 30874 tablet_bootstrap.cc:654] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:35.865599 30873 tablet_bootstrap.cc:654] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:35.866458 30873 log.cc:824] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:35.866480 30874 log.cc:824] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:35.867743 30872 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: No bootstrap required, opened a new log
I20260929 18:44:35.867954 30872 ts_tablet_manager.cc:1397] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:35.868738 30873 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: No bootstrap required, opened a new log
I20260929 18:44:35.868738 30874 tablet_bootstrap.cc:492] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: No bootstrap required, opened a new log
I20260929 18:44:35.868935 30874 ts_tablet_manager.cc:1397] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:35.868935 30873 ts_tablet_manager.cc:1397] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 18:44:35.871887 30872 raft_consensus.cc:348] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.872193 30872 raft_consensus.cc:374] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:35.872259 30872 raft_consensus.cc:729] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90cea4a94304595874d1d20957a60dd, State: Initialized, Role: FOLLOWER
I20260929 18:44:35.872730 30872 consensus_queue.cc:260] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.873431 30724 heartbeater.cc:503] Master 127.11.118.254:42199 was elected leader, sending a full tablet report...
I20260929 18:44:35.873807 30872 ts_tablet_manager.cc:1428] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 18:44:35.873749 30873 raft_consensus.cc:348] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.874001 30873 raft_consensus.cc:374] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:35.874055 30873 raft_consensus.cc:729] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ed945a4ecf4e4397f5952645cad184, State: Initialized, Role: FOLLOWER
I20260929 18:44:35.874532 30873 consensus_queue.cc:260] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.873503 30874 raft_consensus.cc:348] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.874809 30874 raft_consensus.cc:374] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:35.874882 30874 raft_consensus.cc:729] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148cc672146c482da3c4789829ce5f67, State: Initialized, Role: FOLLOWER
I20260929 18:44:35.875310 30874 consensus_queue.cc:260] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:35.875640 30873 ts_tablet_manager.cc:1428] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 18:44:35.876108 30855 heartbeater.cc:503] Master 127.11.118.254:42199 was elected leader, sending a full tablet report...
I20260929 18:44:35.876202 30874 ts_tablet_manager.cc:1428] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260929 18:44:35.876596 30593 heartbeater.cc:503] Master 127.11.118.254:42199 was elected leader, sending a full tablet report...
W20260929 18:44:35.899184 30594 tablet.cc:2369] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:36.045382 30878 raft_consensus.cc:482] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:36.045636 30878 raft_consensus.cc:504] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:36.046852 30878 leader_election.cc:288] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148cc672146c482da3c4789829ce5f67 (127.11.118.195:34851), c5ed945a4ecf4e4397f5952645cad184 (127.11.118.193:36583)
W20260929 18:44:36.050056 30856 tablet.cc:2369] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:36.051066 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.047307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:40399 (local address 127.11.118.193:36583)
0929 18:44:36.047444 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:36.047451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:36.047471 (+ 20us) server_negotiation.cc:407] Connection header received
0929 18:44:36.047585 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:36.047587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:36.047631 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:36.047734 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:36.048054 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.048715 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.049896 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.050487 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.050624 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:36.050676 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:36.050864 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 18:44:36.050932 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:44:36.051066 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.047179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36583 (local address 127.11.118.194:40399)
0929 18:44:36.047332 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:36.047350 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:36.047462 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:36.047748 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:36.047750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:36.047768 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:36.047893 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:36.047901 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.048828 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.048831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:36.049743 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:36.049752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.050618 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.050622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:36.050729 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:36.050747 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 18:44:36.050847 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 18:44:36.050930 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260929 18:44:36.051314 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.047290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:56515 (local address 127.11.118.195:34851)
0929 18:44:36.047430 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:36.047434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:36.047861 (+ 427us) server_negotiation.cc:407] Connection header received
0929 18:44:36.047999 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:36.048002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:36.048060 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:36.048116 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:36.048407 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.048994 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.050143 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.050770 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.050915 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:36.050960 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:36.051140 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 18:44:36.051209 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:44:36.051333 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.047179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:34851 (local address 127.11.118.194:56515)
0929 18:44:36.047782 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:36.047793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:36.047888 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:36.048156 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:36.048159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:36.048176 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:36.048298 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:36.048304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.049111 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.049114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:36.050009 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:36.050019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.050915 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.050918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:36.051020 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:36.051037 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 18:44:36.051125 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 18:44:36.051192 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":241,"thread_start_us":148,"threads_started":1}
I20260929 18:44:36.051928 30547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889f2fbb437447588d482c8b3c98f211" candidate_uuid: "e90cea4a94304595874d1d20957a60dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ed945a4ecf4e4397f5952645cad184" is_pre_election: true
I20260929 18:44:36.052002 30809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889f2fbb437447588d482c8b3c98f211" candidate_uuid: "e90cea4a94304595874d1d20957a60dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148cc672146c482da3c4789829ce5f67" is_pre_election: true
I20260929 18:44:36.052255 30809 raft_consensus.cc:2454] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90cea4a94304595874d1d20957a60dd in term 0.
I20260929 18:44:36.052255 30547 raft_consensus.cc:2454] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90cea4a94304595874d1d20957a60dd in term 0.
I20260929 18:44:36.052750 30614 leader_election.cc:302] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 148cc672146c482da3c4789829ce5f67, c5ed945a4ecf4e4397f5952645cad184, e90cea4a94304595874d1d20957a60dd; no voters:
I20260929 18:44:36.053053 30878 raft_consensus.cc:2781] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:36.053138 30878 raft_consensus.cc:482] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:36.053180 30878 raft_consensus.cc:3037] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:36.054203 30878 raft_consensus.cc:504] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:36.054641 30878 leader_election.cc:288] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [CANDIDATE]: Term 1 election: Requested vote from peers 148cc672146c482da3c4789829ce5f67 (127.11.118.195:34851), c5ed945a4ecf4e4397f5952645cad184 (127.11.118.193:36583)
I20260929 18:44:36.054986 30809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889f2fbb437447588d482c8b3c98f211" candidate_uuid: "e90cea4a94304595874d1d20957a60dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148cc672146c482da3c4789829ce5f67"
I20260929 18:44:36.054998 30547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889f2fbb437447588d482c8b3c98f211" candidate_uuid: "e90cea4a94304595874d1d20957a60dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5ed945a4ecf4e4397f5952645cad184"
I20260929 18:44:36.055091 30809 raft_consensus.cc:3037] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:36.055156 30547 raft_consensus.cc:3037] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:36.056138 30809 raft_consensus.cc:2454] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90cea4a94304595874d1d20957a60dd in term 1.
I20260929 18:44:36.056311 30547 raft_consensus.cc:2454] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90cea4a94304595874d1d20957a60dd in term 1.
I20260929 18:44:36.056466 30614 leader_election.cc:302] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148cc672146c482da3c4789829ce5f67, e90cea4a94304595874d1d20957a60dd; no voters:
I20260929 18:44:36.056639 30878 raft_consensus.cc:2781] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:36.056893 30878 raft_consensus.cc:686] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 1 LEADER]: Becoming Leader. State: Replica: e90cea4a94304595874d1d20957a60dd, State: Running, Role: LEADER
I20260929 18:44:36.057255 30878 consensus_queue.cc:237] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } }
I20260929 18:44:36.060508 30402 catalog_manager.cc:5495] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd reported cstate change: term changed from 0 to 1, leader changed from <none> to e90cea4a94304595874d1d20957a60dd (127.11.118.194). New cstate: current_term: 1 leader_uuid: "e90cea4a94304595874d1d20957a60dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90cea4a94304595874d1d20957a60dd" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 39357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 } health_report { overall_health: UNKNOWN } } }
I20260929 18:44:36.088521 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.084914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.11.118.194:39357)
0929 18:44:36.085053 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:36.085058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:36.085074 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:44:36.085174 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:36.085177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:36.085242 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:36.085351 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:36.085614 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.086115 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.087042 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:36.087270 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:36.087389 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:36.087677 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:36.088073 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:36.088229 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:36.088317 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 18:44:36.088352 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 18:44:36.094682 30547 raft_consensus.cc:1261] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 1 FOLLOWER]: Refusing update from remote peer e90cea4a94304595874d1d20957a60dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:36.094712 30809 raft_consensus.cc:1261] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 1 FOLLOWER]: Refusing update from remote peer e90cea4a94304595874d1d20957a60dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:36.095513 30882 consensus_queue.cc:1035] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "148cc672146c482da3c4789829ce5f67" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 34851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:36.095717 30878 consensus_queue.cc:1035] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5ed945a4ecf4e4397f5952645cad184" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 36583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 18:44:36.098451 30725 tablet.cc:2369] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:36.103189 30886 mvcc.cc:205] Tried to move back new op lower bound from 7334737822075916288 to 7334737821933162496. Current Snapshot: MvccSnapshot[applied={T|T < 7334737822075916288}]
I20260929 18:44:36.103725 30887 mvcc.cc:205] Tried to move back new op lower bound from 7334737822075916288 to 7334737821933162496. Current Snapshot: MvccSnapshot[applied={T|T < 7334737822075916288}]
I20260929 18:44:36.111312 30405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:36.111501 30405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:36.113988 30405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4985fbee2c274af7856889dfd61dd12d: Sending DeleteTablet for 3 replicas of tablet 889f2fbb437447588d482c8b3c98f211
I20260929 18:44:36.114791 30658 tablet_service.cc:1514] Processing DeleteTablet for tablet 889f2fbb437447588d482c8b3c98f211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:36 UTC) from {username='slave'} at 127.0.0.1:53402
I20260929 18:44:36.114794 30527 tablet_service.cc:1514] Processing DeleteTablet for tablet 889f2fbb437447588d482c8b3c98f211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:36 UTC) from {username='slave'} at 127.0.0.1:40500
I20260929 18:44:36.114863 30789 tablet_service.cc:1514] Processing DeleteTablet for tablet 889f2fbb437447588d482c8b3c98f211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:36 UTC) from {username='slave'} at 127.0.0.1:33212
I20260929 18:44:36.115352 30897 tablet_replica.cc:330] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: stopping tablet replica
I20260929 18:44:36.115458 30896 tablet_replica.cc:330] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184: stopping tablet replica
I20260929 18:44:36.115685 30897 raft_consensus.cc:2229] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:36.115736 30896 raft_consensus.cc:2229] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:36.116003 30897 raft_consensus.cc:2258] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:36.116012 30896 raft_consensus.cc:2258] T 889f2fbb437447588d482c8b3c98f211 P c5ed945a4ecf4e4397f5952645cad184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:36.116370 30895 tablet_replica.cc:330] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: stopping tablet replica
I20260929 18:44:36.116444 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30465
I20260929 18:44:36.116618 30895 raft_consensus.cc:2229] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:44:36.117000 30895 raft_consensus.cc:2258] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:36.119040 30895 ts_tablet_manager.cc:1905] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:36.121233 30895 ts_tablet_manager.cc:1918] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:36.121340 30895 log.cc:1196] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/889f2fbb437447588d482c8b3c98f211
I20260929 18:44:36.121676 30895 ts_tablet_manager.cc:1939] T 889f2fbb437447588d482c8b3c98f211 P e90cea4a94304595874d1d20957a60dd: Deleting consensus metadata
I20260929 18:44:36.122898 30897 ts_tablet_manager.cc:1905] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:36.122864 30392 catalog_manager.cc:4841] TS e90cea4a94304595874d1d20957a60dd (127.11.118.194:39357): tablet 889f2fbb437447588d482c8b3c98f211 (table test-table [id=18d871ec82884952a49dfbad88686d8d]) successfully deleted
W20260929 18:44:36.124920 30389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:36583 (error 108)
I20260929 18:44:36.125967 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:36.125670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:36583 (local address 127.0.0.1:40512)
0929 18:44:36.125807 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:36.125879 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:36583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:44:36.126029 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30596
I20260929 18:44:36.126818 30897 ts_tablet_manager.cc:1918] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:36.126921 30897 log.cc:1196] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/889f2fbb437447588d482c8b3c98f211
I20260929 18:44:36.127240 30897 ts_tablet_manager.cc:1939] T 889f2fbb437447588d482c8b3c98f211 P 148cc672146c482da3c4789829ce5f67: Deleting consensus metadata
W20260929 18:44:36.127447 30389 catalog_manager.cc:4584] TS c5ed945a4ecf4e4397f5952645cad184 (127.11.118.193:36583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 889f2fbb437447588d482c8b3c98f211: Network error: Client connection negotiation failed: client connection to 127.11.118.193:36583: connect: Connection refused (error 111)
I20260929 18:44:36.128566 30392 catalog_manager.cc:4841] TS 148cc672146c482da3c4789829ce5f67 (127.11.118.195:34851): tablet 889f2fbb437447588d482c8b3c98f211 (table test-table [id=18d871ec82884952a49dfbad88686d8d]) successfully deleted
I20260929 18:44:36.132589 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30727
I20260929 18:44:36.138310 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30374
2026-09-29T18:44:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1213 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:36 dist-test-slave-cgv7 krb5kdc[30901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:36 dist-test-slave-cgv7 krb5kdc[30901](info): set up 2 sockets
Sep 29 18:44:36 dist-test-slave-cgv7 krb5kdc[30901](info): commencing operation
krb5kdc: starting...
W20260929 18:44:37.178191 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:44:37.306192 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34003
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:37.420940 30914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:37.421206 30914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:37.421247 30914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:37.425238 30914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:37.425302 30914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:37.425321 30914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:37.425338 30914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:37.425354 30914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:37.430785 30914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:37.431859 30914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:37.432833 30914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:37.441692 30922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:37.441720 30919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:37.442128 30914 server_base.cc:1034] running on GCE node
W20260929 18:44:37.441697 30920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:37.442772 30914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:37.443987 30914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:37.445175 30914 hybrid_clock.cc:648] HybridClock initialized: now 1790707477445140 us; error 56 us; skew 500 ppm
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:37.448176 30914 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:37.449152 30914 webserver.cc:460] Webserver started at http://127.11.118.254:46067/ using document root <none> and password file <none>
I20260929 18:44:37.449730 30914 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:37.449795 30914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:37.450039 30914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:37.452206 30914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "647d9b285e884a018dc2181874bcd74b"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "393f954249c74e3abfde478e96432a95"
server_key_iv: "645fc6bed71c716e52dd0348ea2b888f"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.452778 30914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "647d9b285e884a018dc2181874bcd74b"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "393f954249c74e3abfde478e96432a95"
server_key_iv: "645fc6bed71c716e52dd0348ea2b888f"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.456511 30914 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 18:44:37.458899 30929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:37.460073 30914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:44:37.460209 30914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "647d9b285e884a018dc2181874bcd74b"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "393f954249c74e3abfde478e96432a95"
server_key_iv: "645fc6bed71c716e52dd0348ea2b888f"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.460310 30914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:37.488927 30914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:37.492096 30914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:37.492302 30914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:37.499650 30914 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:34003
I20260929 18:44:37.499671 30981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:34003 every 8 connection(s)
I20260929 18:44:37.500604 30914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:37.503024 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 30914
I20260929 18:44:37.503166 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:37.503497 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:37.503497 11739 external_mini_cluster.cc:1428] Setting key 1315bf6863ed641095f46da4bc6900bf
I20260929 18:44:37.509760 30982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: Bootstrap starting.
I20260929 18:44:37.512001 30982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: Neither blocks nor log segments found. Creating new log.
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:37.512781 30982 log.cc:824] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:37.514751 30982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: No bootstrap required, opened a new log
I20260929 18:44:37.517838 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:37.504939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.11.118.254:34003)
0929 18:44:37.505371 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:37.505400 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:37.505435 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:44:37.506210 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:37.506228 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:37.506601 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:37.506933 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:37.507762 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.508760 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.509633 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.510175 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.510372 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:37.512903 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:37.512928 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:37.512948 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:37.512983 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:37.514795 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:37.515398 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:37.515408 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:37.515413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:37.515532 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:37.515858 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:37.515862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:37.515865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:37.516263 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:37.516482 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:37.516781 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 18:44:37.517093 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":145,"threads_started":1}
I20260929 18:44:37.518520 30982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } }
I20260929 18:44:37.518769 30982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:37.518833 30982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647d9b285e884a018dc2181874bcd74b, State: Initialized, Role: FOLLOWER
I20260929 18:44:37.520215 30982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } }
I20260929 18:44:37.520386 30982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:37.520464 30982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:37.520586 30982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:37.521514 30982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } }
I20260929 18:44:37.521844 30982 leader_election.cc:302] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 647d9b285e884a018dc2181874bcd74b; no voters:
I20260929 18:44:37.522145 30982 leader_election.cc:288] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:37.522342 30987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:37.522562 30987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [term 1 LEADER]: Becoming Leader. State: Replica: 647d9b285e884a018dc2181874bcd74b, State: Running, Role: LEADER
I20260929 18:44:37.522925 30987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } }
I20260929 18:44:37.523484 30982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:37.524017 30987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647d9b285e884a018dc2181874bcd74b. Latest consensus state: current_term: 1 leader_uuid: "647d9b285e884a018dc2181874bcd74b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } } }
I20260929 18:44:37.524130 30987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:37.524528 30989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "647d9b285e884a018dc2181874bcd74b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647d9b285e884a018dc2181874bcd74b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34003 } } }
I20260929 18:44:37.524641 30989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:37.524664 30992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:37.527789 30992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:37.533802 30992 catalog_manager.cc:1347] Generated new cluster ID: a3679d844fef4fd3b613c808347c2e57
I20260929 18:44:37.533890 30992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:37.545619 30992 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:37.546705 30992 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:37.556887 30992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: Generated new TSK 0
I20260929 18:44:37.557785 30992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.118.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
Entry for principal kudu/127.11.118.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab.
I20260929 18:44:37.598133 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:37.748581 31008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:37.748860 31008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:37.748915 31008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:37.753168 31008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:37.753247 31008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:37.753346 31008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:37.758984 31008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.193.keytab
--principal=kudu/127.11.118.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:37.760140 31008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:37.761214 31008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:37.770972 31016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:37.770974 31014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:37.771062 31013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:37.771160 31008 server_base.cc:1034] running on GCE node
I20260929 18:44:37.771637 31008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:37.772328 31008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:37.773517 31008 hybrid_clock.cc:648] HybridClock initialized: now 1790707477773501 us; error 29 us; skew 500 ppm
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:37.776338 31008 init.cc:377] Logged in from keytab as kudu/127.11.118.193@KRBTEST.COM (short username kudu)
I20260929 18:44:37.777244 31008 webserver.cc:460] Webserver started at http://127.11.118.193:42841/ using document root <none> and password file <none>
I20260929 18:44:37.777755 31008 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:37.777812 31008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:37.777981 31008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:37.780030 31008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "ccdf1580dac84c6c8f2df45350910fd2"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7cd4a5d26cea07bad42ee4b43cb7945f"
server_key_iv: "65abe6cd7e49c65c49e48ff7008654d1"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.780543 31008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "ccdf1580dac84c6c8f2df45350910fd2"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7cd4a5d26cea07bad42ee4b43cb7945f"
server_key_iv: "65abe6cd7e49c65c49e48ff7008654d1"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.784296 31008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260929 18:44:37.786798 31023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:37.788077 31008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260929 18:44:37.788172 31008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "ccdf1580dac84c6c8f2df45350910fd2"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7cd4a5d26cea07bad42ee4b43cb7945f"
server_key_iv: "65abe6cd7e49c65c49e48ff7008654d1"
server_key_version: "encryptionkey@0"
I20260929 18:44:37.788311 31008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:37.804780 31008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:37.808447 31008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:37.808655 31008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:37.809281 31008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:37.810146 31008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:37.810194 31008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:37.810263 31008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:37.810319 31008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:37.819455 31008 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:41081
I20260929 18:44:37.819471 31136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:41081 every 8 connection(s)
I20260929 18:44:37.820375 31008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:37.826018 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31008
I20260929 18:44:37.826164 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:37.826529 11739 external_mini_cluster.cc:1428] Setting key 56fe8ff846c02d90fe04ce9e169dbe75
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.193@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:37.836190 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:37.822197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:60967 (local address 127.11.118.254:34003)
0929 18:44:37.822394 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:37.822399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:37.823298 (+ 899us) server_negotiation.cc:407] Connection header received
0929 18:44:37.824836 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:37.824838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:37.824888 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:37.824972 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:37.826522 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.827146 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.828654 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.829104 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.829230 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:37.831912 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:37.831915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:37.831917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:37.831941 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:37.833867 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:37.834491 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:37.834498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:37.834502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:37.834603 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:37.834962 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:37.834965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:37.834967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:37.835222 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:37.835327 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:37.835855 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 18:44:37.835991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
WARNING: no policy specified for kudu/127.11.118.194@KRBTEST.COM; defaulting to no policy
I20260929 18:44:37.837067 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:37.822640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34003 (local address 127.11.118.193:60967)
0929 18:44:37.823091 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:37.823144 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:37.824645 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:37.825168 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:37.825184 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:37.825605 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:37.826344 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:37.826367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.827304 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.827309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:37.828524 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:37.828534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:37.829249 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:37.829252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:37.829398 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:37.830052 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:37.830070 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:37.831718 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:37.834020 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:37.834025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:37.834039 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:37.834372 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:37.834706 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:37.834708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:37.834710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:37.834851 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:37.835330 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:37.835335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:37.835574 (+ 239us) client_negotiation.cc:769] Sending connection context
0929 18:44:37.835735 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 18:44:37.836002 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260929 18:44:37.838586 31137 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34003
I20260929 18:44:37.838871 31137 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:37.839424 31137 heartbeater.cc:511] Master 127.11.118.254:34003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.194@KRBTEST.COM" created.
I20260929 18:44:37.841148 30946 ts_manager.cc:194] Registered new tserver with Master: ccdf1580dac84c6c8f2df45350910fd2 (127.11.118.193:41081)
I20260929 18:44:37.842366 30946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.193@KRBTEST.COM'} at 127.11.118.193:60967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
Entry for principal kudu/127.11.118.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab.
I20260929 18:44:37.857506 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:37.971277 31142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:37.971575 31142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:37.971622 31142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:37.975950 31142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:37.976022 31142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:37.976082 31142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:37.981601 31142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.194.keytab
--principal=kudu/127.11.118.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:37.982752 31142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:37.984064 31142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:37.994557 31150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:37.994557 31148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:37.994620 31147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:37.994989 31142 server_base.cc:1034] running on GCE node
I20260929 18:44:37.995406 31142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:37.996071 31142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:37.997247 31142 hybrid_clock.cc:648] HybridClock initialized: now 1790707477997222 us; error 31 us; skew 500 ppm
Sep 29 18:44:37 dist-test-slave-cgv7 krb5kdc[30901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:38.000169 31142 init.cc:377] Logged in from keytab as kudu/127.11.118.194@KRBTEST.COM (short username kudu)
I20260929 18:44:38.001075 31142 webserver.cc:460] Webserver started at http://127.11.118.194:33575/ using document root <none> and password file <none>
I20260929 18:44:38.001654 31142 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:38.001745 31142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:38.001978 31142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:38.004154 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "7a709d6c53ec4168a45814c9c729dd5e"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7a3d53e895b2dfed78ea84974b97b536"
server_key_iv: "aa37afbed4683a0d016b45d856ea6cc9"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.004736 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "7a709d6c53ec4168a45814c9c729dd5e"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7a3d53e895b2dfed78ea84974b97b536"
server_key_iv: "aa37afbed4683a0d016b45d856ea6cc9"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.008613 31142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:44:38.011236 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.012552 31142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:44:38.012699 31142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "7a709d6c53ec4168a45814c9c729dd5e"
format_stamp: "Formatted at 2026-09-29 18:44:37 on dist-test-slave-cgv7"
server_key: "7a3d53e895b2dfed78ea84974b97b536"
server_key_iv: "aa37afbed4683a0d016b45d856ea6cc9"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.012825 31142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:38.028362 31142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:38.031802 31142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:38.032012 31142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:38.032609 31142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:38.033440 31142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:38.033510 31142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.033582 31142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:38.033634 31142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.042541 31142 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:45063
I20260929 18:44:38.042567 31270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:45063 every 8 connection(s)
I20260929 18:44:38.043435 31142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:38.044354 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31142
I20260929 18:44:38.044505 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:38.044771 11739 external_mini_cluster.cc:1428] Setting key 501779c2bf98f5c752c0aebd61bd9f1c
WARNING: no policy specified for kudu/127.11.118.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.195@KRBTEST.COM" created.
Sep 29 18:44:38 dist-test-slave-cgv7 krb5kdc[30901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.194@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:38.064033 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.046385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:36575 (local address 127.11.118.254:34003)
0929 18:44:38.046516 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.046520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.047499 (+ 979us) server_negotiation.cc:407] Connection header received
0929 18:44:38.049063 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.049066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.049123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.049225 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:38.050920 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.051506 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.053444 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.053677 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.053767 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.059715 (+ 5948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:38.059723 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:38.059729 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:38.059768 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:38.061397 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.062063 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.062070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.062075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.062171 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.062625 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.062631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.062636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.062928 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:38.063058 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.063619 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.063779 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 18:44:38.065081 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.046786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34003 (local address 127.11.118.194:36575)
0929 18:44:38.047309 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.047370 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.048829 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.049420 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.049436 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.049974 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:38.050725 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.050749 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.051657 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.051663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.053280 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.053295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.053851 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.053855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.054084 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.054784 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:38.054812 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:38.059482 (+ 4670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:38.061531 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:38.061536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:38.061552 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:38.061921 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:38.062332 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:38.062338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:38.062343 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:38.062491 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:38.063060 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:38.063066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:38.063346 (+ 280us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.063512 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.063951 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":101,"threads_started":1}
I20260929 18:44:38.066466 31271 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34003
I20260929 18:44:38.066718 31271 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:38.067240 31271 heartbeater.cc:511] Master 127.11.118.254:34003 requested a full tablet report, sending...
I20260929 18:44:38.068450 30946 ts_manager.cc:194] Registered new tserver with Master: 7a709d6c53ec4168a45814c9c729dd5e (127.11.118.194:45063)
I20260929 18:44:38.069154 30946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.194@KRBTEST.COM'} at 127.11.118.194:36575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
Entry for principal kudu/127.11.118.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab.
I20260929 18:44:38.078243 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:38.188917 31276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:38.189222 31276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:38.189311 31276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:38.193507 31276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:38.193594 31276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:38.193687 31276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:38.199077 31276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.195.keytab
--principal=kudu/127.11.118.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:38.200166 31276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:38.201193 31276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:38.210827 31281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:38.210768 31282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:38.210788 31284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:38.210870 31276 server_base.cc:1034] running on GCE node
I20260929 18:44:38.211496 31276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:38.212185 31276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:38.213370 31276 hybrid_clock.cc:648] HybridClock initialized: now 1790707478213349 us; error 35 us; skew 500 ppm
Sep 29 18:44:38 dist-test-slave-cgv7 krb5kdc[30901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707478, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:38.216375 31276 init.cc:377] Logged in from keytab as kudu/127.11.118.195@KRBTEST.COM (short username kudu)
I20260929 18:44:38.217363 31276 webserver.cc:460] Webserver started at http://127.11.118.195:34205/ using document root <none> and password file <none>
I20260929 18:44:38.217897 31276 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:38.217955 31276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:38.218115 31276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:38.220173 31276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "cbfd63c57c774daab81469b5dfb669db"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "01edfb57cf55b973e0319f89762ba5f5"
server_key_iv: "210cab284e36a09ec3b68664ac7c372b"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.220679 31276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "cbfd63c57c774daab81469b5dfb669db"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "01edfb57cf55b973e0319f89762ba5f5"
server_key_iv: "210cab284e36a09ec3b68664ac7c372b"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.224998 31276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:44:38.227480 31291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.228925 31276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:44:38.229076 31276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "cbfd63c57c774daab81469b5dfb669db"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "01edfb57cf55b973e0319f89762ba5f5"
server_key_iv: "210cab284e36a09ec3b68664ac7c372b"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.229199 31276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:38.246498 31276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:38.249900 31276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:38.250113 31276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:38.250762 31276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:38.251597 31276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:38.251646 31276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.251685 31276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:38.251726 31276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.260730 31276 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:44953
I20260929 18:44:38.260818 31404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:44953 every 8 connection(s)
I20260929 18:44:38.261664 31276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:38.265255 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31276
I20260929 18:44:38.265406 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:38.265694 11739 external_mini_cluster.cc:1428] Setting key 2bc7d17de57f9359ca1bb5a35c018fdf
Sep 29 18:44:38 dist-test-slave-cgv7 krb5kdc[30901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707478, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.195@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:38.276443 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.263673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:57763 (local address 127.11.118.254:34003)
0929 18:44:38.263816 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.263820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.264685 (+ 865us) server_negotiation.cc:407] Connection header received
0929 18:44:38.266358 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.266362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.266420 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.266497 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:38.267929 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.268457 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.269634 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.269882 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.270010 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.272586 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:38.272590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:38.272593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:38.272627 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:38.274241 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.274836 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.274841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.274842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.274939 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.275309 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.275313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.275315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.275516 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:38.275629 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.276145 (+ 516us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.276278 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:38.277298 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.264089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34003 (local address 127.11.118.195:57763)
0929 18:44:38.264542 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.264582 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.266112 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.266662 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.266671 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.267152 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:38.267645 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.267659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.268590 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.268594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.269502 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.269512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.270049 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.270053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.270242 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.270930 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:44:38.270949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:38.272403 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:38.274413 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:38.274420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:38.274446 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:38.274722 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:38.275048 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:44:38.275056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:44:38.275057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:44:38.275199 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:44:38.275627 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:38.275634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:38.275903 (+ 269us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.276105 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.276461 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260929 18:44:38.278584 31405 heartbeater.cc:347] Connected to a master server at 127.11.118.254:34003
I20260929 18:44:38.278925 31405 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:38.279481 31405 heartbeater.cc:511] Master 127.11.118.254:34003 requested a full tablet report, sending...
I20260929 18:44:38.280548 30946 ts_manager.cc:194] Registered new tserver with Master: cbfd63c57c774daab81469b5dfb669db (127.11.118.195:44953)
I20260929 18:44:38.281090 30946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.118.195@KRBTEST.COM'} at 127.11.118.195:57763
I20260929 18:44:38.292613 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:38.303201 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.295191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.11.118.254:34003)
0929 18:44:38.295373 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.295378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.295601 (+ 223us) server_negotiation.cc:407] Connection header received
0929 18:44:38.295656 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.295658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.295704 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.295778 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:38.296536 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.297107 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.298016 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.298246 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.298450 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.299906 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:38.299909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:38.299911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:38.299933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:38.301313 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.301743 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.301745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.301746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.301875 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:38.302199 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:38.302201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:38.302203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:38.302532 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:38.302641 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.302896 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.302998 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 18:44:38.306417 30946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:38.309626 30946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:38.325829 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.318645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:41081 (local address 127.0.0.1:32890)
0929 18:44:38.319133 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.319190 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.319322 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.320577 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.320584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.320612 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:38.320791 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.320798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.322527 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.322534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.323977 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.323988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.325246 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.325250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.325372 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.325414 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.325523 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.325650 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":100,"threads_started":1}
I20260929 18:44:38.325834 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.319429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.11.118.193:41081)
0929 18:44:38.319987 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.320003 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.320028 (+ 25us) server_negotiation.cc:407] Connection header received
0929 18:44:38.320111 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.320116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.320370 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.321042 (+ 672us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:38.321234 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.322341 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.324143 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.325113 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.325246 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.325440 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.325549 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.325650 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":112,"threads_started":1}
I20260929 18:44:38.327461 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.319413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:44953 (local address 127.0.0.1:52014)
0929 18:44:38.319690 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.319704 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.319794 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.322654 (+ 2860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.322658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.322680 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:38.322783 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.322789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.324490 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.324493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.325654 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.325663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.327084 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.327088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.327185 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.327211 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.327259 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.327323 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":71,"thread_start_us":110,"threads_started":1}
I20260929 18:44:38.327497 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.319863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:45063 (local address 127.0.0.1:44522)
0929 18:44:38.320446 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.320462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.320553 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.323374 (+ 2821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.323376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.323391 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:38.323472 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.323478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.324744 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.324748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.326058 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.326067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.327143 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.327150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.327241 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.327264 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.327316 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.327397 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":168,"thread_start_us":101,"threads_started":1}
I20260929 18:44:38.328063 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.322167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.11.118.194:45063)
0929 18:44:38.322522 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.322535 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.322564 (+ 29us) server_negotiation.cc:407] Connection header received
0929 18:44:38.322657 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.322662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.322860 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.323038 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:38.323606 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.324553 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.326190 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.327005 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.327710 (+ 705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.327839 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.327887 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.327958 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":131,"threads_started":1}
I20260929 18:44:38.328716 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.321695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.11.118.195:44953)
0929 18:44:38.322076 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.322090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.322114 (+ 24us) server_negotiation.cc:407] Connection header received
0929 18:44:38.322190 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.322196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.322443 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.322658 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:38.323110 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.324349 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.325769 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.326946 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.328287 (+ 1341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.328442 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.328496 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.328584 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":201,"threads_started":1}
I20260929 18:44:38.329607 31071 tablet_service.cc:1467] Processing CreateTablet for tablet 82295609523449c2828ceb2b1d557338 (DEFAULT_TABLE table=test-table [id=f147398c86304ef0afae830345cd3d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:38.330829 31071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82295609523449c2828ceb2b1d557338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:38.331101 31205 tablet_service.cc:1467] Processing CreateTablet for tablet 82295609523449c2828ceb2b1d557338 (DEFAULT_TABLE table=test-table [id=f147398c86304ef0afae830345cd3d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:38.331481 31339 tablet_service.cc:1467] Processing CreateTablet for tablet 82295609523449c2828ceb2b1d557338 (DEFAULT_TABLE table=test-table [id=f147398c86304ef0afae830345cd3d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:38.332154 31205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82295609523449c2828ceb2b1d557338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:38.332401 31339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82295609523449c2828ceb2b1d557338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:38.337752 31422 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: Bootstrap starting.
I20260929 18:44:38.339313 31423 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Bootstrap starting.
I20260929 18:44:38.340169 31422 tablet_bootstrap.cc:654] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:38.340503 31424 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Bootstrap starting.
I20260929 18:44:38.341094 31422 log.cc:824] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:38.341604 31423 tablet_bootstrap.cc:654] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:38.342860 31424 tablet_bootstrap.cc:654] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:38.342909 31423 log.cc:824] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:38.343300 31422 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: No bootstrap required, opened a new log
I20260929 18:44:38.343510 31422 ts_tablet_manager.cc:1397] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:38.344013 31424 log.cc:824] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:38.344969 31423 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: No bootstrap required, opened a new log
I20260929 18:44:38.345216 31423 ts_tablet_manager.cc:1397] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 18:44:38.345705 31424 tablet_bootstrap.cc:492] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: No bootstrap required, opened a new log
I20260929 18:44:38.345898 31424 ts_tablet_manager.cc:1397] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:38.348130 31422 raft_consensus.cc:348] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.348430 31422 raft_consensus.cc:374] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:38.348490 31422 raft_consensus.cc:729] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdf1580dac84c6c8f2df45350910fd2, State: Initialized, Role: FOLLOWER
I20260929 18:44:38.348977 31422 consensus_queue.cc:260] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.349210 31423 raft_consensus.cc:348] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.349472 31423 raft_consensus.cc:374] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:38.349535 31423 raft_consensus.cc:729] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfd63c57c774daab81469b5dfb669db, State: Initialized, Role: FOLLOWER
I20260929 18:44:38.349514 31424 raft_consensus.cc:348] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.349788 31424 raft_consensus.cc:374] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:38.349854 31424 raft_consensus.cc:729] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a709d6c53ec4168a45814c9c729dd5e, State: Initialized, Role: FOLLOWER
I20260929 18:44:38.349970 31423 consensus_queue.cc:260] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.350069 31422 ts_tablet_manager.cc:1428] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:44:38.350313 31424 consensus_queue.cc:260] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.350445 31137 heartbeater.cc:503] Master 127.11.118.254:34003 was elected leader, sending a full tablet report...
I20260929 18:44:38.350750 31405 heartbeater.cc:503] Master 127.11.118.254:34003 was elected leader, sending a full tablet report...
I20260929 18:44:38.350999 31423 ts_tablet_manager.cc:1428] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260929 18:44:38.351537 31271 heartbeater.cc:503] Master 127.11.118.254:34003 was elected leader, sending a full tablet report...
I20260929 18:44:38.352367 31424 ts_tablet_manager.cc:1428] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:38.400522 31428 raft_consensus.cc:482] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:38.400743 31428 raft_consensus.cc:504] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.402005 31428 leader_election.cc:288] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a709d6c53ec4168a45814c9c729dd5e (127.11.118.194:45063), cbfd63c57c774daab81469b5dfb669db (127.11.118.195:44953)
I20260929 18:44:38.405917 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.402415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:46275 (local address 127.11.118.195:44953)
0929 18:44:38.402542 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.402547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.402793 (+ 246us) server_negotiation.cc:407] Connection header received
0929 18:44:38.402994 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.402997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.403042 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.403122 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:38.403408 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.403892 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.404913 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.405307 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.405445 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.405496 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.405710 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.405783 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:38.405936 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.402347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:59659 (local address 127.11.118.194:45063)
0929 18:44:38.402574 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.402580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.402662 (+ 82us) server_negotiation.cc:407] Connection header received
0929 18:44:38.402833 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.402841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.402886 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.402989 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:38.403265 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.403765 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.404875 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.405291 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.405433 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.405486 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.405729 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.405803 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260929 18:44:38.405984 31431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.402205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:45063 (local address 127.11.118.193:59659)
0929 18:44:38.402551 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.402566 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.402715 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.402998 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.403001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.403017 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:38.403151 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.403157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.403888 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.403894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.404739 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.404749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.405433 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.405437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.405606 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.405624 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.405713 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.405786 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":44,"thread_start_us":153,"threads_started":1}
I20260929 18:44:38.405992 31432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.402316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:44953 (local address 127.11.118.193:46275)
0929 18:44:38.402691 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.402702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.402829 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.403142 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.403145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.403163 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:38.403287 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.403293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.404001 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.404007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.404774 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.404792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.405436 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.405439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.405554 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.405595 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.405680 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.405753 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":6,"thread_start_us":107,"threads_started":1}
I20260929 18:44:38.406787 31359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82295609523449c2828ceb2b1d557338" candidate_uuid: "ccdf1580dac84c6c8f2df45350910fd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfd63c57c774daab81469b5dfb669db" is_pre_election: true
I20260929 18:44:38.406807 31225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82295609523449c2828ceb2b1d557338" candidate_uuid: "ccdf1580dac84c6c8f2df45350910fd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a709d6c53ec4168a45814c9c729dd5e" is_pre_election: true
I20260929 18:44:38.407121 31225 raft_consensus.cc:2454] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccdf1580dac84c6c8f2df45350910fd2 in term 0.
I20260929 18:44:38.407121 31359 raft_consensus.cc:2454] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccdf1580dac84c6c8f2df45350910fd2 in term 0.
I20260929 18:44:38.407637 31027 leader_election.cc:302] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a709d6c53ec4168a45814c9c729dd5e, ccdf1580dac84c6c8f2df45350910fd2; no voters:
I20260929 18:44:38.407949 31428 raft_consensus.cc:2781] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:38.408038 31428 raft_consensus.cc:482] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:38.408082 31428 raft_consensus.cc:3037] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:38.409111 31428 raft_consensus.cc:504] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.409474 31428 leader_election.cc:288] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [CANDIDATE]: Term 1 election: Requested vote from peers 7a709d6c53ec4168a45814c9c729dd5e (127.11.118.194:45063), cbfd63c57c774daab81469b5dfb669db (127.11.118.195:44953)
I20260929 18:44:38.409847 31225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82295609523449c2828ceb2b1d557338" candidate_uuid: "ccdf1580dac84c6c8f2df45350910fd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a709d6c53ec4168a45814c9c729dd5e"
I20260929 18:44:38.409914 31359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82295609523449c2828ceb2b1d557338" candidate_uuid: "ccdf1580dac84c6c8f2df45350910fd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfd63c57c774daab81469b5dfb669db"
I20260929 18:44:38.409971 31225 raft_consensus.cc:3037] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:38.410072 31359 raft_consensus.cc:3037] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:38.411109 31225 raft_consensus.cc:2454] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccdf1580dac84c6c8f2df45350910fd2 in term 1.
I20260929 18:44:38.411223 31359 raft_consensus.cc:2454] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccdf1580dac84c6c8f2df45350910fd2 in term 1.
I20260929 18:44:38.411437 31027 leader_election.cc:302] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a709d6c53ec4168a45814c9c729dd5e, ccdf1580dac84c6c8f2df45350910fd2; no voters:
I20260929 18:44:38.411624 31428 raft_consensus.cc:2781] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:38.411866 31428 raft_consensus.cc:686] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [term 1 LEADER]: Becoming Leader. State: Replica: ccdf1580dac84c6c8f2df45350910fd2, State: Running, Role: LEADER
I20260929 18:44:38.412205 31428 consensus_queue.cc:237] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } }
I20260929 18:44:38.415470 30946 catalog_manager.cc:5495] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccdf1580dac84c6c8f2df45350910fd2 (127.11.118.193). New cstate: current_term: 1 leader_uuid: "ccdf1580dac84c6c8f2df45350910fd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdf1580dac84c6c8f2df45350910fd2" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 } health_report { overall_health: UNKNOWN } } }
I20260929 18:44:38.449239 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.445565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.11.118.193:41081)
0929 18:44:38.445700 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.445703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.445719 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:44:38.445797 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.445799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.445857 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.445972 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:38.446201 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.446739 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.447632 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.447880 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.447971 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.448328 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:38.448806 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:38.448915 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.449053 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.449114 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:38.455616 31359 raft_consensus.cc:1261] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 1 FOLLOWER]: Refusing update from remote peer ccdf1580dac84c6c8f2df45350910fd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:38.455616 31225 raft_consensus.cc:1261] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 1 FOLLOWER]: Refusing update from remote peer ccdf1580dac84c6c8f2df45350910fd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:38.456395 31433 consensus_queue.cc:1035] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a709d6c53ec4168a45814c9c729dd5e" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:38.456576 31428 consensus_queue.cc:1035] T 82295609523449c2828ceb2b1d557338 P ccdf1580dac84c6c8f2df45350910fd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfd63c57c774daab81469b5dfb669db" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:38.474001 30946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:38.474197 30946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:38.476588 30946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 647d9b285e884a018dc2181874bcd74b: Sending DeleteTablet for 3 replicas of tablet 82295609523449c2828ceb2b1d557338
I20260929 18:44:38.477479 31339 tablet_service.cc:1514] Processing DeleteTablet for tablet 82295609523449c2828ceb2b1d557338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:38 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:52014
I20260929 18:44:38.477480 31205 tablet_service.cc:1514] Processing DeleteTablet for tablet 82295609523449c2828ceb2b1d557338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:38 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:44522
I20260929 18:44:38.477985 31446 tablet_replica.cc:330] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: stopping tablet replica
I20260929 18:44:38.478015 31447 tablet_replica.cc:330] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: stopping tablet replica
I20260929 18:44:38.478294 31071 tablet_service.cc:1514] Processing DeleteTablet for tablet 82295609523449c2828ceb2b1d557338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:38 UTC) from {username='kudu', principal='kudu/127.11.118.254@KRBTEST.COM'} at 127.0.0.1:32890
I20260929 18:44:38.478364 31447 raft_consensus.cc:2229] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:38.478391 31446 raft_consensus.cc:2229] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:38.478672 31447 raft_consensus.cc:2258] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:38.478873 31446 raft_consensus.cc:2258] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:38.479074 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31008
I20260929 18:44:38.481235 31446 ts_tablet_manager.cc:1905] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:38.481721 31447 ts_tablet_manager.cc:1905] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:38.484090 31447 ts_tablet_manager.cc:1918] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:38.484234 31447 log.cc:1196] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/82295609523449c2828ceb2b1d557338
I20260929 18:44:38.484616 31447 ts_tablet_manager.cc:1939] T 82295609523449c2828ceb2b1d557338 P 7a709d6c53ec4168a45814c9c729dd5e: Deleting consensus metadata
I20260929 18:44:38.484807 31446 ts_tablet_manager.cc:1918] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:38.484905 31446 log.cc:1196] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/82295609523449c2828ceb2b1d557338
I20260929 18:44:38.485221 31446 ts_tablet_manager.cc:1939] T 82295609523449c2828ceb2b1d557338 P cbfd63c57c774daab81469b5dfb669db: Deleting consensus metadata
I20260929 18:44:38.486057 30933 catalog_manager.cc:4841] TS 7a709d6c53ec4168a45814c9c729dd5e (127.11.118.194:45063): tablet 82295609523449c2828ceb2b1d557338 (table test-table [id=f147398c86304ef0afae830345cd3d6d]) successfully deleted
I20260929 18:44:38.486459 30932 catalog_manager.cc:4841] TS cbfd63c57c774daab81469b5dfb669db (127.11.118.195:44953): tablet 82295609523449c2828ceb2b1d557338 (table test-table [id=f147398c86304ef0afae830345cd3d6d]) successfully deleted
W20260929 18:44:38.486814 30932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:41081 (error 108)
I20260929 18:44:38.486936 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31142
I20260929 18:44:38.488938 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.488562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:41081 (local address 127.0.0.1:32900)
0929 18:44:38.488738 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.488820 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 18:44:38.489161 30932 catalog_manager.cc:4584] TS ccdf1580dac84c6c8f2df45350910fd2 (127.11.118.193:41081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82295609523449c2828ceb2b1d557338: Network error: Client connection negotiation failed: client connection to 127.11.118.193:41081: connect: Connection refused (error 111)
I20260929 18:44:38.493155 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31276
I20260929 18:44:38.499285 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 30914
2026-09-29T18:44:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2361 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T18:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:38Z Disabled control of system clock
I20260929 18:44:38.529858 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40877
--webserver_interface=127.11.118.254
--webserver_port=0
--builtin_ntp_servers=127.11.118.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:40877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:38.640671 31453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:38.640965 31453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:38.641002 31453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:38.645079 31453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:38.645150 31453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:38.645166 31453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:38.645183 31453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:38.645203 31453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:38.650044 31453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:40877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:38.651186 31453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:38.652515 31453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:38.660921 31458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:38.660953 31459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:38.661144 31453 server_base.cc:1034] running on GCE node
W20260929 18:44:38.660925 31461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:38.661863 31453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:38.663000 31453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:38.664192 31453 hybrid_clock.cc:648] HybridClock initialized: now 1790707478664170 us; error 34 us; skew 500 ppm
I20260929 18:44:38.666239 31453 webserver.cc:460] Webserver started at http://127.11.118.254:37617/ using document root <none> and password file <none>
I20260929 18:44:38.666967 31453 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:38.667060 31453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:38.667261 31453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:38.669411 31453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "eed9f3c73e5643b88c3621d8708e3eeb"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "df06e17b2c56ec54a0ef2f4a76591cbf"
server_key_iv: "2278b4608bcda70f4b8ea1314661d932"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.669991 31453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "eed9f3c73e5643b88c3621d8708e3eeb"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "df06e17b2c56ec54a0ef2f4a76591cbf"
server_key_iv: "2278b4608bcda70f4b8ea1314661d932"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.673923 31453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 18:44:38.676702 31467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.678115 31453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:44:38.678342 31453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "eed9f3c73e5643b88c3621d8708e3eeb"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "df06e17b2c56ec54a0ef2f4a76591cbf"
server_key_iv: "2278b4608bcda70f4b8ea1314661d932"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.678484 31453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:38.711087 31453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:38.711890 31453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:38.712100 31453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:38.718999 31453 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:40877
I20260929 18:44:38.719012 31519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:40877 every 8 connection(s)
I20260929 18:44:38.719976 31453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:38.723124 31520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:38.726581 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31453
I20260929 18:44:38.726727 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:38.727039 11739 external_mini_cluster.cc:1428] Setting key f52ccb51067cc67e8ac505605c733695
I20260929 18:44:38.729519 31520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: Bootstrap starting.
I20260929 18:44:38.732164 31520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:38.732987 31520 log.cc:824] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:38.735205 31520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: No bootstrap required, opened a new log
I20260929 18:44:38.737077 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.728613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39188 (local address 127.11.118.254:40877)
0929 18:44:38.729362 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.729378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.729422 (+ 44us) server_negotiation.cc:407] Connection header received
0929 18:44:38.730194 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.730215 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.730721 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.731085 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:38.731548 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.732596 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.733568 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.734069 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.734262 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.735247 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:38.735252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:38.735265 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:38.735729 (+ 464us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:38.735788 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:38.735811 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:38.735924 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.736070 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.736344 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":156,"threads_started":1}
I20260929 18:44:38.739290 31520 raft_consensus.cc:348] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } }
I20260929 18:44:38.739589 31520 raft_consensus.cc:374] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:38.739652 31520 raft_consensus.cc:729] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed9f3c73e5643b88c3621d8708e3eeb, State: Initialized, Role: FOLLOWER
I20260929 18:44:38.740048 31520 consensus_queue.cc:260] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } }
I20260929 18:44:38.740168 31520 raft_consensus.cc:388] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:38.740212 31520 raft_consensus.cc:482] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:38.740285 31520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:38.741417 31520 raft_consensus.cc:504] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } }
I20260929 18:44:38.741798 31520 leader_election.cc:302] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eed9f3c73e5643b88c3621d8708e3eeb; no voters:
I20260929 18:44:38.742132 31520 leader_election.cc:288] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:38.742259 31525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:38.742563 31525 raft_consensus.cc:686] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [term 1 LEADER]: Becoming Leader. State: Replica: eed9f3c73e5643b88c3621d8708e3eeb, State: Running, Role: LEADER
I20260929 18:44:38.742888 31525 consensus_queue.cc:237] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } }
I20260929 18:44:38.743474 31520 sys_catalog.cc:564] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:38.744712 31526 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } } }
I20260929 18:44:38.744872 31526 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:38.744848 31527 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader eed9f3c73e5643b88c3621d8708e3eeb. Latest consensus state: current_term: 1 leader_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed9f3c73e5643b88c3621d8708e3eeb" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40877 } } }
I20260929 18:44:38.744916 31527 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:38.745340 31534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:38.748878 31534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:38.755743 31534 catalog_manager.cc:1347] Generated new cluster ID: 40444391e35349aba24736f2ba39b151
I20260929 18:44:38.755838 31534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:38.767330 31534 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:38.768297 31534 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:38.773885 31534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: Generated new TSK 0
I20260929 18:44:38.774752 31534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:38.786031 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.193:0
--local_ip_for_outbound_sockets=127.11.118.193
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--builtin_ntp_servers=127.11.118.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 18:44:38.897857 31544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:38.898190 31544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:38.898312 31544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:38.902417 31544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:38.902514 31544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:38.902612 31544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.193
I20260929 18:44:38.907555 31544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.118.193
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.193
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:38.908711 31544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:38.910171 31544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:38.920346 31550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:38.920382 31552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:38.920341 31549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:38.920672 31544 server_base.cc:1034] running on GCE node
I20260929 18:44:38.921115 31544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:38.921801 31544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:38.922997 31544 hybrid_clock.cc:648] HybridClock initialized: now 1790707478922980 us; error 29 us; skew 500 ppm
I20260929 18:44:38.924887 31544 webserver.cc:460] Webserver started at http://127.11.118.193:33849/ using document root <none> and password file <none>
I20260929 18:44:38.925530 31544 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:38.925626 31544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:38.925851 31544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:38.928042 31544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data/instance:
uuid: "b04ab3ced2754e17b9ccb69bf00950ce"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "b922d58f3f5109606026a66b905a282f"
server_key_iv: "c9bbdb35e11bc33d6338fef2d6b404a3"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.928623 31544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance:
uuid: "b04ab3ced2754e17b9ccb69bf00950ce"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "b922d58f3f5109606026a66b905a282f"
server_key_iv: "c9bbdb35e11bc33d6338fef2d6b404a3"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.932679 31544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 18:44:38.935256 31558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.936522 31544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:38.936661 31544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal
uuid: "b04ab3ced2754e17b9ccb69bf00950ce"
format_stamp: "Formatted at 2026-09-29 18:44:38 on dist-test-slave-cgv7"
server_key: "b922d58f3f5109606026a66b905a282f"
server_key_iv: "c9bbdb35e11bc33d6338fef2d6b404a3"
server_key_version: "encryptionkey@0"
I20260929 18:44:38.936780 31544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:38.949754 31544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:38.950534 31544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:38.950737 31544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:38.951341 31544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:38.952270 31544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:38.952346 31544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.952418 31544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:38.952468 31544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:38.961390 31544 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.193:37161
I20260929 18:44:38.961406 31671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.193:37161 every 8 connection(s)
I20260929 18:44:38.962365 31544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/data/info.pb
I20260929 18:44:38.962666 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31544
I20260929 18:44:38.962859 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-0/wal/instance
I20260929 18:44:38.963292 11739 external_mini_cluster.cc:1428] Setting key 9308ffa5157b234a4a0c8c41ba700205
I20260929 18:44:38.965441 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.194:0
--local_ip_for_outbound_sockets=127.11.118.194
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--builtin_ntp_servers=127.11.118.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:38.973075 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.964258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.193:52819 (local address 127.11.118.254:40877)
0929 18:44:38.964382 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:38.964386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:38.965631 (+ 1245us) server_negotiation.cc:407] Connection header received
0929 18:44:38.967614 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:38.967618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:38.967670 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:38.967752 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:38.968639 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.969527 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.970832 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.971271 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.971405 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.972485 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:38.972491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:38.972496 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:38.972516 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:38.972544 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:38.972557 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:38.972645 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:38.972782 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 18:44:38.972866 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:44:38.973809 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:38.964711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40877 (local address 127.11.118.193:52819)
0929 18:44:38.965470 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:38.965529 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:38.967402 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:38.967927 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:38.967933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:38.968018 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:38.968500 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.968512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.969673 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.969676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.970702 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:38.970711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:38.971435 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:38.971438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:38.971576 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:38.972211 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:38.972213 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:38.972267 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:38.972330 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:38.972648 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:38.972655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:38.972659 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 18:44:38.972751 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 18:44:38.972965 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260929 18:44:38.975072 31672 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40877
I20260929 18:44:38.975332 31672 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:38.976019 31672 heartbeater.cc:511] Master 127.11.118.254:40877 requested a full tablet report, sending...
I20260929 18:44:38.977648 31483 ts_manager.cc:194] Registered new tserver with Master: b04ab3ced2754e17b9ccb69bf00950ce (127.11.118.193:37161)
I20260929 18:44:38.978863 31483 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.193:52819
W20260929 18:44:39.082545 31675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:39.082865 31675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:39.082943 31675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:39.087370 31675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:39.087432 31675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:39.087519 31675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.194
I20260929 18:44:39.092444 31675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.118.194
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.194
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:39.093571 31675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:39.095000 31675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:39.104934 31683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:39.104956 31681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:39.105093 31675 server_base.cc:1034] running on GCE node
W20260929 18:44:39.104934 31680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:39.105618 31675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:39.106361 31675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:39.107560 31675 hybrid_clock.cc:648] HybridClock initialized: now 1790707479107538 us; error 32 us; skew 500 ppm
I20260929 18:44:39.109956 31675 webserver.cc:460] Webserver started at http://127.11.118.194:37353/ using document root <none> and password file <none>
I20260929 18:44:39.110642 31675 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:39.110744 31675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:39.110965 31675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:39.113139 31675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data/instance:
uuid: "61a5c78444b849a2aa1d9cfda9378ab3"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "c5e9bbb8068b25504f102f677017e68b"
server_key_iv: "88fbe739095f45b92bdc987d5ae8e8ae"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.113709 31675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance:
uuid: "61a5c78444b849a2aa1d9cfda9378ab3"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "c5e9bbb8068b25504f102f677017e68b"
server_key_iv: "88fbe739095f45b92bdc987d5ae8e8ae"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.117655 31675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:39.120489 31689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.121899 31675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:44:39.122038 31675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal
uuid: "61a5c78444b849a2aa1d9cfda9378ab3"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "c5e9bbb8068b25504f102f677017e68b"
server_key_iv: "88fbe739095f45b92bdc987d5ae8e8ae"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.122159 31675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:39.149252 31675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:39.150152 31675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:39.150422 31675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:39.151201 31675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:39.152281 31675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:39.152356 31675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.152449 31675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:39.152504 31675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.161645 31675 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.194:34595
I20260929 18:44:39.161664 31802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.194:34595 every 8 connection(s)
I20260929 18:44:39.162592 31675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/data/info.pb
I20260929 18:44:39.163422 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31675
I20260929 18:44:39.163556 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal/instance
I20260929 18:44:39.163903 11739 external_mini_cluster.cc:1428] Setting key efc391922ca10f7a653a054d5a3dcca1
I20260929 18:44:39.166150 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.118.195:0
--local_ip_for_outbound_sockets=127.11.118.195
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--builtin_ntp_servers=127.11.118.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 18:44:39.178206 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.164576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.194:49397 (local address 127.11.118.254:40877)
0929 18:44:39.164727 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.164730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.165952 (+ 1222us) server_negotiation.cc:407] Connection header received
0929 18:44:39.167553 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.167556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.167601 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.167670 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:39.168848 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.169411 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.170712 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.170941 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.171058 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.177578 (+ 6520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:39.177582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:39.177584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:39.177601 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:39.177622 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:39.177631 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:39.177738 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.177966 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.178046 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:44:39.179193 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.165039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40877 (local address 127.11.118.194:49397)
0929 18:44:39.165807 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.165849 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.167295 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.167869 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.167880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.168004 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:39.168683 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.168699 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.169561 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.169566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.170574 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.170585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.171101 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.171105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.171276 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.177192 (+ 5916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:39.177194 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:39.177248 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:39.177337 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:39.177758 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:39.177771 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:39.177784 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.177933 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.178319 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":168,"threads_started":1}
I20260929 18:44:39.180526 31803 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40877
I20260929 18:44:39.180779 31803 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:39.181335 31803 heartbeater.cc:511] Master 127.11.118.254:40877 requested a full tablet report, sending...
I20260929 18:44:39.182493 31483 ts_manager.cc:194] Registered new tserver with Master: 61a5c78444b849a2aa1d9cfda9378ab3 (127.11.118.194:34595)
I20260929 18:44:39.183099 31483 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.194:49397
W20260929 18:44:39.278589 31806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:39.278901 31806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:39.278983 31806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:39.283293 31806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:39.283380 31806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:39.283466 31806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.118.195
I20260929 18:44:39.288589 31806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.118.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.118.195
--webserver_port=0
--tserver_master_addrs=127.11.118.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.118.195
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:39.289839 31806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:39.291368 31806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:39.301403 31814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:39.301403 31811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:39.301393 31812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:39.301656 31806 server_base.cc:1034] running on GCE node
I20260929 18:44:39.302156 31806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:39.302909 31806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:39.304100 31806 hybrid_clock.cc:648] HybridClock initialized: now 1790707479304083 us; error 30 us; skew 500 ppm
I20260929 18:44:39.306006 31806 webserver.cc:460] Webserver started at http://127.11.118.195:46355/ using document root <none> and password file <none>
I20260929 18:44:39.306680 31806 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:39.306774 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:39.306988 31806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:39.309165 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data/instance:
uuid: "88df605b17864dd9a4d84f9360314380"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "dec0dd97d8d4b82dabff88c22d13a100"
server_key_iv: "5c43e97ce03d2ba6fa4413554bd9d416"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.309803 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance:
uuid: "88df605b17864dd9a4d84f9360314380"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "dec0dd97d8d4b82dabff88c22d13a100"
server_key_iv: "5c43e97ce03d2ba6fa4413554bd9d416"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.313997 31806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:39.316821 31820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.318198 31806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:44:39.318388 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal
uuid: "88df605b17864dd9a4d84f9360314380"
format_stamp: "Formatted at 2026-09-29 18:44:39 on dist-test-slave-cgv7"
server_key: "dec0dd97d8d4b82dabff88c22d13a100"
server_key_iv: "5c43e97ce03d2ba6fa4413554bd9d416"
server_key_version: "encryptionkey@0"
I20260929 18:44:39.318536 31806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:39.331987 31806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:39.332741 31806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:39.332958 31806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:39.333570 31806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 18:44:39.334609 31806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 18:44:39.334681 31806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.334753 31806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 18:44:39.334808 31806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:39.344067 31806 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.195:46379
I20260929 18:44:39.344094 31933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.195:46379 every 8 connection(s)
I20260929 18:44:39.345275 31806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/data/info.pb
I20260929 18:44:39.353217 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31806
I20260929 18:44:39.353327 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal/instance
I20260929 18:44:39.353628 11739 external_mini_cluster.cc:1428] Setting key f4eaf7bdf2fe920781d5a2e807398b2a
I20260929 18:44:39.355363 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.347281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:40735 (local address 127.11.118.254:40877)
0929 18:44:39.347445 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.347449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.348611 (+ 1162us) server_negotiation.cc:407] Connection header received
0929 18:44:39.350033 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.350036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.350088 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.350171 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:39.351262 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.351804 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.353053 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.353326 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.353456 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.354816 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:39.354829 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:39.354835 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:39.354853 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:39.354881 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:39.354891 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:39.354952 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.355156 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.355229 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:44:39.356482 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.347667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:40877 (local address 127.11.118.195:40735)
0929 18:44:39.348420 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.348472 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.349796 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.350412 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.350419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.350514 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:44:39.351046 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.351059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.351990 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.351992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.352882 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.352890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.353504 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.353508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.353721 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.354493 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 18:44:39.354495 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:44:39.354537 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 18:44:39.354634 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:44:39.354980 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:44:39.354990 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:44:39.354997 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.355137 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.355484 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":133,"threads_started":1}
I20260929 18:44:39.357697 31934 heartbeater.cc:347] Connected to a master server at 127.11.118.254:40877
I20260929 18:44:39.357921 31934 heartbeater.cc:464] Registering TS with master...
I20260929 18:44:39.358497 31934 heartbeater.cc:511] Master 127.11.118.254:40877 requested a full tablet report, sending...
I20260929 18:44:39.359524 31483 ts_manager.cc:194] Registered new tserver with Master: 88df605b17864dd9a4d84f9360314380 (127.11.118.195:46379)
I20260929 18:44:39.360074 31483 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.118.195:40735
I20260929 18:44:39.368438 11739 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 18:44:39.374450 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.370813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.11.118.254:40877)
0929 18:44:39.370990 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.370994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.371076 (+ 82us) server_negotiation.cc:407] Connection header received
0929 18:44:39.371169 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.371171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.371213 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.371294 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:39.371606 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.372162 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.373009 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.373219 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.373295 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.374042 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:39.374045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:39.374047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:39.374060 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:39.374079 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:39.374087 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:39.374146 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.374217 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.374319 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 18:44:39.377705 31483 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 18:44:39.380950 31483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 18:44:39.396664 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.391406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.11.118.194:34595)
0929 18:44:39.391806 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.391818 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.392228 (+ 410us) server_negotiation.cc:407] Connection header received
0929 18:44:39.392310 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.392315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.392510 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.392662 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:39.392890 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.394015 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.395334 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.395948 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396096 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.396291 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.396388 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.396489 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":95,"threads_started":1}
I20260929 18:44:39.396660 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.391213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:34595 (local address 127.0.0.1:37500)
0929 18:44:39.392029 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.392045 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.392132 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.392669 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.392672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.392691 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:39.392768 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.392774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.394144 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.394151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.395217 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.395229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.396080 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.396191 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.396228 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.396344 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.396490 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":24,"thread_start_us":90,"threads_started":1}
I20260929 18:44:39.397043 31948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.390901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37161 (local address 127.0.0.1:40578)
0929 18:44:39.391323 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.391386 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.391528 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.392158 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.392163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.392186 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:39.392333 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.392340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.393826 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.393829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.395128 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.395139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.396496 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.396673 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.396737 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.396819 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.396907 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":105,"spinlock_wait_cycles":768,"thread_start_us":92,"threads_started":1}
I20260929 18:44:39.397507 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.390396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.195:46379 (local address 127.0.0.1:38618)
0929 18:44:39.391323 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.391376 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.391526 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.392328 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.392331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.392346 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:39.392451 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.392456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.393598 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.393601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.395327 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.395335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.396526 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.396693 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.397230 (+ 537us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.397311 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.397380 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":653,"spinlock_wait_cycles":2816,"thread_start_us":102,"threads_started":1}
I20260929 18:44:39.397816 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.390546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.11.118.193:37161)
0929 18:44:39.391102 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.391116 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.391681 (+ 565us) server_negotiation.cc:407] Connection header received
0929 18:44:39.391763 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.391768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.391986 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.392164 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:39.392473 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.393709 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.395270 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.396111 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396239 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.396428 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.397231 (+ 803us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.397660 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":129,"threads_started":1}
I20260929 18:44:39.398576 31946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.390529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.11.118.195:46379)
0929 18:44:39.390907 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.390920 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.391674 (+ 754us) server_negotiation.cc:407] Connection header received
0929 18:44:39.391763 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.391768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.391993 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.392155 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:39.392546 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.393456 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.395460 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.396228 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.396340 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.396478 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.397362 (+ 884us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.397498 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":119,"threads_started":1}
I20260929 18:44:39.399930 31737 tablet_service.cc:1467] Processing CreateTablet for tablet fdd3e9a395f546ef94af3a17a8923592 (DEFAULT_TABLE table=test-table [id=141779456ac842d1ab0332b76c018d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:39.401039 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd3e9a395f546ef94af3a17a8923592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:39.400857 31868 tablet_service.cc:1467] Processing CreateTablet for tablet fdd3e9a395f546ef94af3a17a8923592 (DEFAULT_TABLE table=test-table [id=141779456ac842d1ab0332b76c018d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:39.401799 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd3e9a395f546ef94af3a17a8923592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:39.402309 31606 tablet_service.cc:1467] Processing CreateTablet for tablet fdd3e9a395f546ef94af3a17a8923592 (DEFAULT_TABLE table=test-table [id=141779456ac842d1ab0332b76c018d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 18:44:39.403291 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd3e9a395f546ef94af3a17a8923592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:39.407764 31951 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Bootstrap starting.
I20260929 18:44:39.409359 31952 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Bootstrap starting.
I20260929 18:44:39.409972 31951 tablet_bootstrap.cc:654] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:39.410852 31953 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Bootstrap starting.
I20260929 18:44:39.410890 31951 log.cc:824] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:39.411741 31952 tablet_bootstrap.cc:654] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:39.412576 31952 log.cc:824] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:39.413146 31953 tablet_bootstrap.cc:654] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:39.413192 31951 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: No bootstrap required, opened a new log
I20260929 18:44:39.413398 31951 ts_tablet_manager.cc:1397] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:39.414019 31953 log.cc:824] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:39.414608 31952 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: No bootstrap required, opened a new log
I20260929 18:44:39.414799 31952 ts_tablet_manager.cc:1397] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 18:44:39.416042 31953 tablet_bootstrap.cc:492] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: No bootstrap required, opened a new log
I20260929 18:44:39.416219 31953 ts_tablet_manager.cc:1397] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 18:44:39.417892 31951 raft_consensus.cc:348] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.418164 31951 raft_consensus.cc:374] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:39.418211 31951 raft_consensus.cc:729] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88df605b17864dd9a4d84f9360314380, State: Initialized, Role: FOLLOWER
I20260929 18:44:39.418229 31952 raft_consensus.cc:348] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.418618 31952 raft_consensus.cc:374] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:39.418677 31952 raft_consensus.cc:729] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a5c78444b849a2aa1d9cfda9378ab3, State: Initialized, Role: FOLLOWER
I20260929 18:44:39.418704 31951 consensus_queue.cc:260] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.419378 31952 consensus_queue.cc:260] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.419788 31951 ts_tablet_manager.cc:1428] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 18:44:39.420116 31803 heartbeater.cc:503] Master 127.11.118.254:40877 was elected leader, sending a full tablet report...
I20260929 18:44:39.420207 31934 heartbeater.cc:503] Master 127.11.118.254:40877 was elected leader, sending a full tablet report...
I20260929 18:44:39.420356 31952 ts_tablet_manager.cc:1428] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 18:44:39.420347 31953 raft_consensus.cc:348] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.420581 31953 raft_consensus.cc:374] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:39.420626 31953 raft_consensus.cc:729] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04ab3ced2754e17b9ccb69bf00950ce, State: Initialized, Role: FOLLOWER
I20260929 18:44:39.421030 31953 consensus_queue.cc:260] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.421743 31672 heartbeater.cc:503] Master 127.11.118.254:40877 was elected leader, sending a full tablet report...
I20260929 18:44:39.421980 31953 ts_tablet_manager.cc:1428] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260929 18:44:39.463634 31673 tablet.cc:2369] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:44:39.596308 31935 tablet.cc:2369] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 18:44:39.663754 31804 tablet.cc:2369] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 18:44:39.695808 31957 raft_consensus.cc:482] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 18:44:39.696096 31957 raft_consensus.cc:504] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.697255 31957 leader_election.cc:288] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b04ab3ced2754e17b9ccb69bf00950ce (127.11.118.193:37161), 61a5c78444b849a2aa1d9cfda9378ab3 (127.11.118.194:34595)
I20260929 18:44:39.702728 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.697705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:58397 (local address 127.11.118.193:37161)
0929 18:44:39.697846 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.697852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.698164 (+ 312us) server_negotiation.cc:407] Connection header received
0929 18:44:39.698300 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.698303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.698375 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.698472 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:39.698768 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.699490 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.701389 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.701997 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.702126 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.702178 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.702511 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.702587 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 18:44:39.702728 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.697693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.118.195:51683 (local address 127.11.118.194:34595)
0929 18:44:39.697839 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.697844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.697996 (+ 152us) server_negotiation.cc:407] Connection header received
0929 18:44:39.698068 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.698075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.698141 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.698231 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 18:44:39.698593 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.699431 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.701389 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.701997 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.702127 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.702178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.702511 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.702587 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:44:39.702741 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.697576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.194:34595 (local address 127.11.118.195:51683)
0929 18:44:39.697847 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.697864 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.697978 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.698293 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.698297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.698316 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:39.698448 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.698460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.699599 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.699603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.701238 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.701249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.702141 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.702145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.702350 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.702399 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.702483 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.702592 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":260}
I20260929 18:44:39.702749 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.697576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37161 (local address 127.11.118.195:58397)
0929 18:44:39.698039 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.698052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:44:39.698163 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:44:39.698497 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:44:39.698499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:44:39.698517 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 18:44:39.698652 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.698657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.699604 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.699606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.701238 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:44:39.701249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.702143 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.702146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:44:39.702393 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.702410 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 18:44:39.702492 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 18:44:39.702592 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":293,"thread_start_us":134,"threads_started":1}
I20260929 18:44:39.703548 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd3e9a395f546ef94af3a17a8923592" candidate_uuid: "88df605b17864dd9a4d84f9360314380" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" is_pre_election: true
I20260929 18:44:39.703696 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd3e9a395f546ef94af3a17a8923592" candidate_uuid: "88df605b17864dd9a4d84f9360314380" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" is_pre_election: true
I20260929 18:44:39.703857 31757 raft_consensus.cc:2454] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88df605b17864dd9a4d84f9360314380 in term 0.
I20260929 18:44:39.704038 31626 raft_consensus.cc:2454] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88df605b17864dd9a4d84f9360314380 in term 0.
I20260929 18:44:39.704339 31822 leader_election.cc:302] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a5c78444b849a2aa1d9cfda9378ab3, 88df605b17864dd9a4d84f9360314380; no voters:
I20260929 18:44:39.704630 31957 raft_consensus.cc:2781] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 18:44:39.704718 31957 raft_consensus.cc:482] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 18:44:39.704761 31957 raft_consensus.cc:3037] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:39.705772 31957 raft_consensus.cc:504] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.706128 31957 leader_election.cc:288] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [CANDIDATE]: Term 1 election: Requested vote from peers b04ab3ced2754e17b9ccb69bf00950ce (127.11.118.193:37161), 61a5c78444b849a2aa1d9cfda9378ab3 (127.11.118.194:34595)
I20260929 18:44:39.706489 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd3e9a395f546ef94af3a17a8923592" candidate_uuid: "88df605b17864dd9a4d84f9360314380" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04ab3ced2754e17b9ccb69bf00950ce"
I20260929 18:44:39.706523 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd3e9a395f546ef94af3a17a8923592" candidate_uuid: "88df605b17864dd9a4d84f9360314380" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a5c78444b849a2aa1d9cfda9378ab3"
I20260929 18:44:39.706656 31757 raft_consensus.cc:3037] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:39.706653 31626 raft_consensus.cc:3037] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:39.707743 31757 raft_consensus.cc:2454] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88df605b17864dd9a4d84f9360314380 in term 1.
I20260929 18:44:39.707746 31626 raft_consensus.cc:2454] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88df605b17864dd9a4d84f9360314380 in term 1.
I20260929 18:44:39.708130 31822 leader_election.cc:302] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a5c78444b849a2aa1d9cfda9378ab3, 88df605b17864dd9a4d84f9360314380; no voters:
I20260929 18:44:39.708360 31957 raft_consensus.cc:2781] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:39.708601 31957 raft_consensus.cc:686] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 1 LEADER]: Becoming Leader. State: Replica: 88df605b17864dd9a4d84f9360314380, State: Running, Role: LEADER
I20260929 18:44:39.708949 31957 consensus_queue.cc:237] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } }
I20260929 18:44:39.712249 31483 catalog_manager.cc:5495] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88df605b17864dd9a4d84f9360314380 (127.11.118.195). New cstate: current_term: 1 leader_uuid: "88df605b17864dd9a4d84f9360314380" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88df605b17864dd9a4d84f9360314380" member_type: VOTER last_known_addr { host: "127.11.118.195" port: 46379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 } health_report { overall_health: UNKNOWN } } }
I20260929 18:44:39.767599 31946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.763928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.11.118.195:46379)
0929 18:44:39.764089 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:39.764093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:39.764107 (+ 14us) server_negotiation.cc:407] Connection header received
0929 18:44:39.764191 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:39.764193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:39.764245 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:39.764359 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:44:39.764650 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.765197 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.766049 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:39.766329 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:39.766431 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:39.766723 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:44:39.767199 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:44:39.767301 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:39.767366 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 18:44:39.767419 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 18:44:39.773816 31757 raft_consensus.cc:1261] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 1 FOLLOWER]: Refusing update from remote peer 88df605b17864dd9a4d84f9360314380: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:39.773814 31626 raft_consensus.cc:1261] T fdd3e9a395f546ef94af3a17a8923592 P b04ab3ced2754e17b9ccb69bf00950ce [term 1 FOLLOWER]: Refusing update from remote peer 88df605b17864dd9a4d84f9360314380: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:44:39.774560 31961 consensus_queue.cc:1035] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b04ab3ced2754e17b9ccb69bf00950ce" member_type: VOTER last_known_addr { host: "127.11.118.193" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:39.774737 31957 consensus_queue.cc:1035] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a5c78444b849a2aa1d9cfda9378ab3" member_type: VOTER last_known_addr { host: "127.11.118.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:44:39.781937 31966 mvcc.cc:205] Tried to move back new op lower bound from 7334737837145620480 to 7334737836890460160. Current Snapshot: MvccSnapshot[applied={T|T < 7334737837145620480}]
I20260929 18:44:39.790486 31482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:39.790645 31482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 18:44:39.793121 31482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eed9f3c73e5643b88c3621d8708e3eeb: Sending DeleteTablet for 3 replicas of tablet fdd3e9a395f546ef94af3a17a8923592
I20260929 18:44:39.793893 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd3e9a395f546ef94af3a17a8923592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:39 UTC) from {username='slave'} at 127.0.0.1:38618
I20260929 18:44:39.793915 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd3e9a395f546ef94af3a17a8923592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:39 UTC) from {username='slave'} at 127.0.0.1:37500
I20260929 18:44:39.793916 31606 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd3e9a395f546ef94af3a17a8923592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 18:44:39 UTC) from {username='slave'} at 127.0.0.1:40578
I20260929 18:44:39.794667 31974 tablet_replica.cc:330] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: stopping tablet replica
I20260929 18:44:39.795004 31974 raft_consensus.cc:2229] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 18:44:39.795341 31974 raft_consensus.cc:2258] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:39.795411 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31544
I20260929 18:44:39.795727 31976 tablet_replica.cc:330] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: stopping tablet replica
I20260929 18:44:39.796006 31976 raft_consensus.cc:2229] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 1 LEADER]: Raft consensus shutting down.
I20260929 18:44:39.796423 31976 raft_consensus.cc:2258] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 18:44:39.797616 31974 ts_tablet_manager.cc:1905] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:39.797843 31976 ts_tablet_manager.cc:1905] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 18:44:39.800594 31974 ts_tablet_manager.cc:1918] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:39.800702 31974 log.cc:1196] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-1/wal/wals/fdd3e9a395f546ef94af3a17a8923592
I20260929 18:44:39.800962 31976 ts_tablet_manager.cc:1918] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 18:44:39.801020 31974 ts_tablet_manager.cc:1939] T fdd3e9a395f546ef94af3a17a8923592 P 61a5c78444b849a2aa1d9cfda9378ab3: Deleting consensus metadata
I20260929 18:44:39.801070 31976 log.cc:1196] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Deleting WAL directory at /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790707223158922-11739-0/minicluster-data/ts-2/wal/wals/fdd3e9a395f546ef94af3a17a8923592
I20260929 18:44:39.801347 31976 ts_tablet_manager.cc:1939] T fdd3e9a395f546ef94af3a17a8923592 P 88df605b17864dd9a4d84f9360314380: Deleting consensus metadata
I20260929 18:44:39.802348 31468 catalog_manager.cc:4841] TS 88df605b17864dd9a4d84f9360314380 (127.11.118.195:46379): tablet fdd3e9a395f546ef94af3a17a8923592 (table test-table [id=141779456ac842d1ab0332b76c018d5f]) successfully deleted
I20260929 18:44:39.802575 31469 catalog_manager.cc:4841] TS 61a5c78444b849a2aa1d9cfda9378ab3 (127.11.118.194:34595): tablet fdd3e9a395f546ef94af3a17a8923592 (table test-table [id=141779456ac842d1ab0332b76c018d5f]) successfully deleted
W20260929 18:44:39.806486 31470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.118.193:37161 (error 108)
I20260929 18:44:39.806665 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31675
I20260929 18:44:39.807981 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:39.807638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.193:37161 (local address 127.0.0.1:40588)
0929 18:44:39.807821 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 18:44:39.807891 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260929 18:44:39.808161 31470 catalog_manager.cc:4584] TS b04ab3ced2754e17b9ccb69bf00950ce (127.11.118.193:37161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdd3e9a395f546ef94af3a17a8923592: Network error: Client connection negotiation failed: client connection to 127.11.118.193:37161: connect: Connection refused (error 111)
I20260929 18:44:39.812481 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31806
I20260929 18:44:39.818657 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31453
2026-09-29T18:44:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1319 ms)
[----------] 6 tests from EncryptionPolicyTest (10910 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:39 dist-test-slave-cgv7 krb5kdc[31980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:39 dist-test-slave-cgv7 krb5kdc[31980](info): set up 2 sockets
Sep 29 18:44:39 dist-test-slave-cgv7 krb5kdc[31980](info): commencing operation
krb5kdc: starting...
W20260929 18:44:40.858583 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:40 dist-test-slave-cgv7 krb5kdc[31980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:40.983501 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:41.095065 31993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:41.095404 31993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:41.095494 31993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:41.099601 31993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:41.099710 31993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:41.099746 31993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:41.099790 31993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:41.099843 31993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:41.105408 31993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39953
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:41.106670 31993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:41.107752 31993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:41.116609 32001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:41.116721 31993 server_base.cc:1034] running on GCE node
W20260929 18:44:41.116595 31998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:41.116600 31999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:41.117496 31993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:41.118810 31993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:41.120033 31993 hybrid_clock.cc:648] HybridClock initialized: now 1790707481120003 us; error 43 us; skew 500 ppm
Sep 29 18:44:41 dist-test-slave-cgv7 krb5kdc[31980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707481, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:41.122937 31993 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:41.123860 31993 webserver.cc:460] Webserver started at http://127.0.0.1:45791/ using document root <none> and password file <none>
I20260929 18:44:41.124387 31993 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:41.124444 31993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:41.124600 31993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:41.126852 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "391130fe4fee4f8f9262815ff14bbaab"
format_stamp: "Formatted at 2026-09-29 18:44:41 on dist-test-slave-cgv7"
server_key: "57ea5165b1614e96b26ae51d09c6aca8"
server_key_iv: "5d35bc53e38b8c0d5aaa4e5bdbbfec7a"
server_key_version: "encryptionkey@0"
I20260929 18:44:41.127682 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "391130fe4fee4f8f9262815ff14bbaab"
format_stamp: "Formatted at 2026-09-29 18:44:41 on dist-test-slave-cgv7"
server_key: "57ea5165b1614e96b26ae51d09c6aca8"
server_key_iv: "5d35bc53e38b8c0d5aaa4e5bdbbfec7a"
server_key_version: "encryptionkey@0"
I20260929 18:44:41.131501 31993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:44:41.133869 32008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:41.135217 31993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:41.135332 31993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "391130fe4fee4f8f9262815ff14bbaab"
format_stamp: "Formatted at 2026-09-29 18:44:41 on dist-test-slave-cgv7"
server_key: "57ea5165b1614e96b26ae51d09c6aca8"
server_key_iv: "5d35bc53e38b8c0d5aaa4e5bdbbfec7a"
server_key_version: "encryptionkey@0"
I20260929 18:44:41.135422 31993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:41.148489 31993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:41.152046 31993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:41.152212 31993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:41.159338 31993 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39953
I20260929 18:44:41.159365 32060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39953 every 8 connection(s)
I20260929 18:44:41.160290 31993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:41.163076 32061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:41.168401 32061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab: Bootstrap starting.
I20260929 18:44:41.170039 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 31993
I20260929 18:44:41.170193 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:41.170506 11739 external_mini_cluster.cc:1428] Setting key 7dc07b4f9b4b64bc9840cf3723ec8682
I20260929 18:44:41.171001 32061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:41.171866 32061 log.cc:824] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:41.173761 32061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab: No bootstrap required, opened a new log
I20260929 18:44:41.177065 32061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } }
I20260929 18:44:41.177309 32061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:41.177372 32061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 391130fe4fee4f8f9262815ff14bbaab, State: Initialized, Role: FOLLOWER
I20260929 18:44:41.177865 32061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } }
I20260929 18:44:41.178038 32061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:41.178102 32061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:41.178215 32061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 0 FOLLOWER]: Advancing to term 1
Sep 29 18:44:41 dist-test-slave-cgv7 krb5kdc[31980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 18:44:41.179335 32061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } }
I20260929 18:44:41.179661 32061 leader_election.cc:302] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 391130fe4fee4f8f9262815ff14bbaab; no voters:
I20260929 18:44:41.180028 32061 leader_election.cc:288] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:41.180137 32066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:41.180660 32066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [term 1 LEADER]: Becoming Leader. State: Replica: 391130fe4fee4f8f9262815ff14bbaab, State: Running, Role: LEADER
I20260929 18:44:41.181006 32066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } }
I20260929 18:44:41.181490 32061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:41.182793 32068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 391130fe4fee4f8f9262815ff14bbaab. Latest consensus state: current_term: 1 leader_uuid: "391130fe4fee4f8f9262815ff14bbaab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } } }
I20260929 18:44:41.182801 32067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "391130fe4fee4f8f9262815ff14bbaab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391130fe4fee4f8f9262815ff14bbaab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39953 } } }
I20260929 18:44:41.182941 32067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:41.182941 32068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:41.183372 32075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:41.186379 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:41.171635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34703 (local address 127.0.0.1:39953)
0929 18:44:41.172111 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:41.172126 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:41.172160 (+ 34us) server_negotiation.cc:407] Connection header received
0929 18:44:41.172965 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:41.173020 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:41.173350 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:41.173677 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:41.174522 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:41.175486 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:41.176362 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:41.176826 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:41.176993 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:41.180024 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:41.180030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:41.180048 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:41.180080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:41.182146 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:41.182885 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:41.182891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:41.182896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:41.183030 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:41.183546 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:41.183552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:41.183557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:41.184067 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:41.184956 (+ 889us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:41.185146 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 18:44:41.185393 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":134,"threads_started":1}
I20260929 18:44:41.187587 32075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:41.189425 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:41.194026 32075 catalog_manager.cc:1347] Generated new cluster ID: 5c900ea014f64b2491540e8f9175b0b5
I20260929 18:44:41.194096 32075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:41.200409 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:41.191984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46941 (local address 127.0.0.1:39953)
0929 18:44:41.192165 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:41.192171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:41.192270 (+ 99us) server_negotiation.cc:407] Connection header received
0929 18:44:41.192412 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:41.192415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:41.192449 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:41.192543 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:41.193191 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:41.193733 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:41.194836 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:41.195193 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:41.195306 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:41.196875 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:41.196880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:41.196883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:41.196910 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:41.198640 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:41.199130 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:41.199134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:41.199136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:41.199226 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:41.199536 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:41.199539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:41.199541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:41.199745 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:41.199889 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:41.200134 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 18:44:41.200231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 18:44:41.210469 32075 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:41.211488 32075 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:41.222155 32075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 391130fe4fee4f8f9262815ff14bbaab: Generated new TSK 0
I20260929 18:44:41.223017 32075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:41.228232 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 31993
W20260929 18:44:41.233798 31978 connection.cc:535] client connection to 127.0.0.1:39953 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39953): Connection reset by peer (error 104)
2026-09-29T18:44:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:41 dist-test-slave-cgv7 krb5kdc[32092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:41 dist-test-slave-cgv7 krb5kdc[32092](info): set up 2 sockets
Sep 29 18:44:41 dist-test-slave-cgv7 krb5kdc[32092](info): commencing operation
krb5kdc: starting...
W20260929 18:44:42.270588 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:42 dist-test-slave-cgv7 krb5kdc[32092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:42.402016 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:42.513900 32105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:42.514175 32105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:42.514222 32105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:42.518302 32105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:42.518370 32105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:42.518400 32105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:42.518462 32105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:42.518494 32105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:42.524155 32105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:42.525307 32105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:42.526448 32105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:42.535333 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:42.535359 32105 server_base.cc:1034] running on GCE node
W20260929 18:44:42.535319 32113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:42.535316 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:42.536213 32105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:42.537377 32105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:42.538581 32105 hybrid_clock.cc:648] HybridClock initialized: now 1790707482538555 us; error 38 us; skew 500 ppm
Sep 29 18:44:42 dist-test-slave-cgv7 krb5kdc[32092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:42.541388 32105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:42.542230 32105 webserver.cc:460] Webserver started at http://127.0.0.1:44063/ using document root <none> and password file <none>
I20260929 18:44:42.542820 32105 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:42.542891 32105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:42.543056 32105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:42.545109 32105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "bf85524f779c4aefb4e7596f194795f4"
format_stamp: "Formatted at 2026-09-29 18:44:42 on dist-test-slave-cgv7"
server_key: "8295f93c0e57247892e93ae613e0631d"
server_key_iv: "6ec6e7799b9dea29065c1c2840b69e91"
server_key_version: "encryptionkey@0"
I20260929 18:44:42.545648 32105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "bf85524f779c4aefb4e7596f194795f4"
format_stamp: "Formatted at 2026-09-29 18:44:42 on dist-test-slave-cgv7"
server_key: "8295f93c0e57247892e93ae613e0631d"
server_key_iv: "6ec6e7799b9dea29065c1c2840b69e91"
server_key_version: "encryptionkey@0"
I20260929 18:44:42.549628 32105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 18:44:42.552361 32120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:42.553699 32105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 18:44:42.553812 32105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "bf85524f779c4aefb4e7596f194795f4"
format_stamp: "Formatted at 2026-09-29 18:44:42 on dist-test-slave-cgv7"
server_key: "8295f93c0e57247892e93ae613e0631d"
server_key_iv: "6ec6e7799b9dea29065c1c2840b69e91"
server_key_version: "encryptionkey@0"
I20260929 18:44:42.553954 32105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:42.573900 32105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:42.577461 32105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:42.577636 32105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:42.585362 32105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34113
I20260929 18:44:42.585484 32172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34113 every 8 connection(s)
I20260929 18:44:42.586555 32105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:42.588644 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32105
I20260929 18:44:42.588843 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:42.589190 11739 external_mini_cluster.cc:1428] Setting key a8bfd316247d0e52b8c310cc39ca4937
I20260929 18:44:42.590024 32173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:42.597330 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4: Bootstrap starting.
I20260929 18:44:42.600109 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:42.590816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52481 (local address 127.0.0.1:34113)
0929 18:44:42.591288 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:42.591308 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:42.591352 (+ 44us) server_negotiation.cc:407] Connection header received
0929 18:44:42.592252 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:42.592284 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:42.592732 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:42.593120 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:42.593556 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:42.595051 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:42.596113 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:42.596841 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:42.597013 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:42.597996 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:42.598002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:42.598019 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:42.598572 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:42.598647 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:42.598680 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:42.598821 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:42.599019 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 18:44:42.599160 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":114,"threads_started":1}
I20260929 18:44:42.600203 32173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:42.600946 32173 log.cc:824] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:42.603309 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4: No bootstrap required, opened a new log
I20260929 18:44:42.603608 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:42.607360 32173 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260929 18:44:42.607596 32173 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:42.607671 32173 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf85524f779c4aefb4e7596f194795f4, State: Initialized, Role: FOLLOWER
I20260929 18:44:42.608115 32173 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260929 18:44:42.608237 32173 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:42.608330 32173 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:42.608458 32173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:42.609380 32173 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260929 18:44:42.609743 32173 leader_election.cc:302] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf85524f779c4aefb4e7596f194795f4; no voters:
I20260929 18:44:42.610101 32173 leader_election.cc:288] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:42.610173 32183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:42.610366 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:42.605917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37851 (local address 127.0.0.1:34113)
0929 18:44:42.606040 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:42.606044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:42.606152 (+ 108us) server_negotiation.cc:407] Connection header received
0929 18:44:42.606327 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:42.606331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:42.606386 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:42.606474 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:42.606756 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:42.607347 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:42.608380 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:42.608703 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:42.608822 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:42.609741 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:42.609745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:42.609747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:42.609761 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:42.609793 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:42.609804 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:42.609908 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:42.610030 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 18:44:42.610068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:44:42.610447 32183 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [term 1 LEADER]: Becoming Leader. State: Replica: bf85524f779c4aefb4e7596f194795f4, State: Running, Role: LEADER
I20260929 18:44:42.610738 32183 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260929 18:44:42.611519 32173 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:42.611923 32183 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf85524f779c4aefb4e7596f194795f4. Latest consensus state: current_term: 1 leader_uuid: "bf85524f779c4aefb4e7596f194795f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260929 18:44:42.612035 32183 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:42.612221 32183 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf85524f779c4aefb4e7596f194795f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf85524f779c4aefb4e7596f194795f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260929 18:44:42.612382 32187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:42.612401 32183 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:42.615657 32187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:42.622141 32187 catalog_manager.cc:1347] Generated new cluster ID: e596129d66c34312abd06b75e70c543d
I20260929 18:44:42.622222 32187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:42.631177 32187 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:42.632396 32187 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:42.646392 32187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf85524f779c4aefb4e7596f194795f4: Generated new TSK 0
I20260929 18:44:42.647305 32187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:42.657379 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32105
W20260929 18:44:42.662928 32090 connection.cc:535] client connection to 127.0.0.1:34113 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34113): Connection reset by peer (error 104)
2026-09-29T18:44:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:42 dist-test-slave-cgv7 krb5kdc[32204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:42 dist-test-slave-cgv7 krb5kdc[32204](info): set up 2 sockets
Sep 29 18:44:42 dist-test-slave-cgv7 krb5kdc[32204](info): commencing operation
krb5kdc: starting...
W20260929 18:44:43.710646 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:43 dist-test-slave-cgv7 krb5kdc[32204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:43.838693 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:43.952699 32217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:43.953027 32217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:43.953112 32217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:43.957240 32217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:43.957361 32217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:43.957403 32217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:43.957456 32217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:43.957494 32217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:43.962880 32217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:43.964022 32217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:43.965058 32217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:43.973783 32222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:43.973788 32225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:43.973903 32217 server_base.cc:1034] running on GCE node
W20260929 18:44:43.973806 32223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:43.974704 32217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:43.975934 32217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:43.977151 32217 hybrid_clock.cc:648] HybridClock initialized: now 1790707483977129 us; error 38 us; skew 500 ppm
Sep 29 18:44:43 dist-test-slave-cgv7 krb5kdc[32204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707483, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:43.979988 32217 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:43.980937 32217 webserver.cc:460] Webserver started at http://127.0.0.1:39997/ using document root <none> and password file <none>
I20260929 18:44:43.981501 32217 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:43.981593 32217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:43.981828 32217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:43.983930 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "f3bf3a48e019400883af7324c1e57951"
format_stamp: "Formatted at 2026-09-29 18:44:43 on dist-test-slave-cgv7"
server_key: "03e3bdca276de07fb771edfd122ea2d7"
server_key_iv: "09a263da6c72dbb685200fa29a326e1f"
server_key_version: "encryptionkey@0"
I20260929 18:44:43.984481 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "f3bf3a48e019400883af7324c1e57951"
format_stamp: "Formatted at 2026-09-29 18:44:43 on dist-test-slave-cgv7"
server_key: "03e3bdca276de07fb771edfd122ea2d7"
server_key_iv: "09a263da6c72dbb685200fa29a326e1f"
server_key_version: "encryptionkey@0"
I20260929 18:44:43.988193 32217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260929 18:44:43.990638 32232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:43.991931 32217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:43.992101 32217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f3bf3a48e019400883af7324c1e57951"
format_stamp: "Formatted at 2026-09-29 18:44:43 on dist-test-slave-cgv7"
server_key: "03e3bdca276de07fb771edfd122ea2d7"
server_key_iv: "09a263da6c72dbb685200fa29a326e1f"
server_key_version: "encryptionkey@0"
I20260929 18:44:43.992225 32217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:43.996994 32217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:44.000445 32217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:44.000674 32217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:44.008546 32217 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44999
I20260929 18:44:44.008569 32284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44999 every 8 connection(s)
I20260929 18:44:44.009554 32217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:44.012362 32285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:44.015376 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32217
I20260929 18:44:44.015523 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:44.015806 11739 external_mini_cluster.cc:1428] Setting key 29c997e00d47ca559d5bc7d7380488fd
I20260929 18:44:44.018540 32285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951: Bootstrap starting.
I20260929 18:44:44.020982 32285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:44.021307 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:44.017294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48313 (local address 127.0.0.1:44999)
0929 18:44:44.017789 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:44.017807 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:44.017843 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:44:44.018548 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:44.018566 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:44.018910 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:44.019248 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:44.020077 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:44.020082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:44.020094 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:44.020104 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:44.020157 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:44.020176 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:44.020333 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:44.020404 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 18:44:44.020564 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":147,"threads_started":1}
I20260929 18:44:44.021793 32285 log.cc:824] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:44.024047 32285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951: No bootstrap required, opened a new log
I20260929 18:44:44.024276 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:44.028463 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:44.026722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43771 (local address 127.0.0.1:44999)
0929 18:44:44.026864 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:44.026870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:44.027126 (+ 256us) server_negotiation.cc:407] Connection header received
0929 18:44:44.027248 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:44.027250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:44.027285 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:44.027350 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:44.028106 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:44.028109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:44.028111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:44.028121 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:44.028144 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:44.028155 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:44.028223 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:44.028332 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 18:44:44.028364 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 18:44:44.029191 32285 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } }
I20260929 18:44:44.029508 32285 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:44.029572 32285 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bf3a48e019400883af7324c1e57951, State: Initialized, Role: FOLLOWER
I20260929 18:44:44.030061 32285 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } }
I20260929 18:44:44.030184 32285 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:44.030230 32285 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:44.030361 32285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:44.031302 32285 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } }
I20260929 18:44:44.031601 32285 leader_election.cc:302] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3bf3a48e019400883af7324c1e57951; no voters:
I20260929 18:44:44.031915 32285 leader_election.cc:288] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:44.032038 32295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:44.032253 32295 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [term 1 LEADER]: Becoming Leader. State: Replica: f3bf3a48e019400883af7324c1e57951, State: Running, Role: LEADER
I20260929 18:44:44.032605 32295 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } }
I20260929 18:44:44.033324 32285 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:44.034312 32297 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3bf3a48e019400883af7324c1e57951. Latest consensus state: current_term: 1 leader_uuid: "f3bf3a48e019400883af7324c1e57951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } } }
I20260929 18:44:44.034459 32297 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:44.034855 32304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:44.034830 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3bf3a48e019400883af7324c1e57951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf3a48e019400883af7324c1e57951" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44999 } } }
I20260929 18:44:44.035048 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:44.038153 32304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:44.044329 32304 catalog_manager.cc:1347] Generated new cluster ID: 915d443ce92d4c12aa2136a62a4c8b9e
I20260929 18:44:44.044481 32304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:44.068477 32304 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:44.068866 32304 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:44.078358 32304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3bf3a48e019400883af7324c1e57951: Generated new TSK 0
I20260929 18:44:44.079162 32304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:44.089579 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32217
2026-09-29T18:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:44 dist-test-slave-cgv7 krb5kdc[32316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:44 dist-test-slave-cgv7 krb5kdc[32316](info): set up 2 sockets
Sep 29 18:44:44 dist-test-slave-cgv7 krb5kdc[32316](info): commencing operation
krb5kdc: starting...
W20260929 18:44:45.130630 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:45.261375 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:45.375377 32329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:45.375706 32329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:45.375783 32329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:45.379890 32329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:45.379998 32329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:45.380039 32329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:45.380101 32329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:45.380143 32329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:45.385524 32329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:45.386670 32329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:45.387758 32329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:45.396497 32335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:45.396515 32334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:45.396509 32337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:45.396567 32329 server_base.cc:1034] running on GCE node
I20260929 18:44:45.397368 32329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:45.398571 32329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:45.399741 32329 hybrid_clock.cc:648] HybridClock initialized: now 1790707485399727 us; error 28 us; skew 500 ppm
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:45.408756 32329 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:45.409931 32329 webserver.cc:460] Webserver started at http://127.0.0.1:42807/ using document root <none> and password file <none>
I20260929 18:44:45.410611 32329 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:45.410717 32329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:45.410969 32329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:45.413126 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "986a3ced7e124682bbd79522ad3f76d8"
format_stamp: "Formatted at 2026-09-29 18:44:45 on dist-test-slave-cgv7"
server_key: "679ca035d42257dce5257c23894470a3"
server_key_iv: "27264cf31e77737979505815a6ba9989"
server_key_version: "encryptionkey@0"
I20260929 18:44:45.413714 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "986a3ced7e124682bbd79522ad3f76d8"
format_stamp: "Formatted at 2026-09-29 18:44:45 on dist-test-slave-cgv7"
server_key: "679ca035d42257dce5257c23894470a3"
server_key_iv: "27264cf31e77737979505815a6ba9989"
server_key_version: "encryptionkey@0"
I20260929 18:44:45.417680 32329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:45.420403 32344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:45.421869 32329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:44:45.422053 32329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "986a3ced7e124682bbd79522ad3f76d8"
format_stamp: "Formatted at 2026-09-29 18:44:45 on dist-test-slave-cgv7"
server_key: "679ca035d42257dce5257c23894470a3"
server_key_iv: "27264cf31e77737979505815a6ba9989"
server_key_version: "encryptionkey@0"
I20260929 18:44:45.422186 32329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:45.447119 32329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:45.450706 32329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:45.450942 32329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:45.458982 32329 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39245
I20260929 18:44:45.458999 32396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39245 every 8 connection(s)
I20260929 18:44:45.460006 32329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:45.463167 32397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:45.468438 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32329
I20260929 18:44:45.468611 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:45.468909 11739 external_mini_cluster.cc:1428] Setting key 4db68a1ffe087df6cf0f5609a36e5a89
I20260929 18:44:45.469586 32397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: Bootstrap starting.
I20260929 18:44:45.472589 32397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:45.473573 32397 log.cc:824] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:45.475852 32397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: No bootstrap required, opened a new log
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 18:44:45.480466 32397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260929 18:44:45.480773 32397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:45.480873 32397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986a3ced7e124682bbd79522ad3f76d8, State: Initialized, Role: FOLLOWER
I20260929 18:44:45.481408 32397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260929 18:44:45.481577 32397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:45.481652 32397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:45.481760 32397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:45.482999 32397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260929 18:44:45.483381 32397 leader_election.cc:302] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 986a3ced7e124682bbd79522ad3f76d8; no voters:
I20260929 18:44:45.483764 32397 leader_election.cc:288] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:45.483875 32402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:45.484196 32402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [term 1 LEADER]: Becoming Leader. State: Replica: 986a3ced7e124682bbd79522ad3f76d8, State: Running, Role: LEADER
I20260929 18:44:45.484581 32402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260929 18:44:45.485103 32397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:45.485376 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:45.470483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35289 (local address 127.0.0.1:39245)
0929 18:44:45.470996 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:45.471011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:45.471047 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:44:45.472094 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:45.472124 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:45.472528 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:45.472862 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:45.473819 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:45.475046 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:45.476216 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:45.476703 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:45.476830 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:45.479721 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:45.479726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:45.479740 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:45.479773 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:45.481903 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:45.482387 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:45.482394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:45.482400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:45.482544 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:45.482819 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:45.482826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:45.482831 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:45.483381 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:45.483576 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:45.483951 (+ 375us) server_negotiation.cc:299] Negotiation successful
0929 18:44:45.484265 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":146,"threads_started":1}
I20260929 18:44:45.486418 32404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "986a3ced7e124682bbd79522ad3f76d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260929 18:44:45.486544 32404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:45.486826 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 986a3ced7e124682bbd79522ad3f76d8. Latest consensus state: current_term: 1 leader_uuid: "986a3ced7e124682bbd79522ad3f76d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986a3ced7e124682bbd79522ad3f76d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260929 18:44:45.486923 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:45.488732 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260929 18:44:45.490978 32418 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:44:45.491056 32418 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:44:45.491115 32411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:45.491963 32411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:45.498114 32411 catalog_manager.cc:1347] Generated new cluster ID: 3902d23fa67a4b719547680122d2d09b
I20260929 18:44:45.498179 32411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:45.502584 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:45.490824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41711 (local address 127.0.0.1:39245)
0929 18:44:45.490949 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:45.490953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:45.494098 (+ 3145us) server_negotiation.cc:407] Connection header received
0929 18:44:45.494217 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:45.494220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:45.494286 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:45.494380 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:45.495208 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:45.495904 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:45.496914 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:45.497158 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:45.497274 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:45.498924 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:45.498927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:45.498930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:45.498956 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:45.500627 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:45.501100 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:45.501104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:45.501106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:45.501195 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:45.501475 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:45.501479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:45.501480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:45.501693 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:45.501822 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:45.502249 (+ 427us) server_negotiation.cc:299] Negotiation successful
0929 18:44:45.502413 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 18:44:45.505097 32411 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:45.506157 32411 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:45.515625 32411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 986a3ced7e124682bbd79522ad3f76d8: Generated new TSK 0
I20260929 18:44:45.516378 32411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:45.521629 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32329
2026-09-29T18:44:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32428](info): set up 2 sockets
Sep 29 18:44:45 dist-test-slave-cgv7 krb5kdc[32428](info): commencing operation
krb5kdc: starting...
W20260929 18:44:46.562647 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:46 dist-test-slave-cgv7 krb5kdc[32428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:46.689982 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:46.800976 32441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:46.801258 32441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:46.801301 32441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:46.805395 32441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:46.805532 32441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:46.805586 32441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:46.805606 32441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:46.805650 32441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:46.811065 32441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:46.812183 32441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:46.813257 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:46.822156 32446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:46.822178 32447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:46.822326 32441 server_base.cc:1034] running on GCE node
W20260929 18:44:46.822530 32449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:46.823094 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:46.824245 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:46.825446 32441 hybrid_clock.cc:648] HybridClock initialized: now 1790707486825432 us; error 26 us; skew 500 ppm
Sep 29 18:44:46 dist-test-slave-cgv7 krb5kdc[32428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707486, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:46.828294 32441 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:46.829157 32441 webserver.cc:460] Webserver started at http://127.0.0.1:45473/ using document root <none> and password file <none>
I20260929 18:44:46.829696 32441 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:46.829768 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:46.829936 32441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:46.832087 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "df063989ff284996a6709e94485bf4e0"
format_stamp: "Formatted at 2026-09-29 18:44:46 on dist-test-slave-cgv7"
server_key: "beb2e1ff228c75b573f129b328ffe0d0"
server_key_iv: "e4d493d93fc707cef4fcb6cec61b238c"
server_key_version: "encryptionkey@0"
I20260929 18:44:46.832628 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "df063989ff284996a6709e94485bf4e0"
format_stamp: "Formatted at 2026-09-29 18:44:46 on dist-test-slave-cgv7"
server_key: "beb2e1ff228c75b573f129b328ffe0d0"
server_key_iv: "e4d493d93fc707cef4fcb6cec61b238c"
server_key_version: "encryptionkey@0"
I20260929 18:44:46.836596 32441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260929 18:44:46.839143 32456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:46.840518 32441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 18:44:46.840631 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "df063989ff284996a6709e94485bf4e0"
format_stamp: "Formatted at 2026-09-29 18:44:46 on dist-test-slave-cgv7"
server_key: "beb2e1ff228c75b573f129b328ffe0d0"
server_key_iv: "e4d493d93fc707cef4fcb6cec61b238c"
server_key_version: "encryptionkey@0"
I20260929 18:44:46.840777 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:46.861866 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:46.865473 32441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:46.865716 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:46.873627 32441 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38491
I20260929 18:44:46.873643 32508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38491 every 8 connection(s)
I20260929 18:44:46.874732 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:46.876837 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32441
I20260929 18:44:46.876971 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:46.877312 11739 external_mini_cluster.cc:1428] Setting key 9498cbd508a65f9f59db039902d5cafa
I20260929 18:44:46.877817 32509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:46.884552 32509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0: Bootstrap starting.
I20260929 18:44:46.886838 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:46.878875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45389 (local address 127.0.0.1:38491)
0929 18:44:46.879482 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:46.879498 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:46.879538 (+ 40us) server_negotiation.cc:407] Connection header received
0929 18:44:46.880215 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:46.880233 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:46.880591 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:46.880992 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:46.881447 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:46.882544 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:46.883472 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:46.883968 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:46.884144 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:46.885066 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:46.885071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:46.885084 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:46.885484 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:46.885538 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:46.885561 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:46.885675 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:46.885806 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 18:44:46.886037 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":166,"threads_started":1}
I20260929 18:44:46.887010 32509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:46.887894 32509 log.cc:824] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:46.889761 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:46.889992 32509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0: No bootstrap required, opened a new log
I20260929 18:44:46.893688 32509 raft_consensus.cc:348] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260929 18:44:46.893941 32509 raft_consensus.cc:374] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:46.894012 32509 raft_consensus.cc:729] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df063989ff284996a6709e94485bf4e0, State: Initialized, Role: FOLLOWER
I20260929 18:44:46.894522 32509 consensus_queue.cc:260] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260929 18:44:46.894690 32509 raft_consensus.cc:388] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:46.894770 32509 raft_consensus.cc:482] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:46.894891 32509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:46.896019 32509 raft_consensus.cc:504] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260929 18:44:46.896371 32509 leader_election.cc:302] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df063989ff284996a6709e94485bf4e0; no voters:
I20260929 18:44:46.896705 32509 leader_election.cc:288] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:46.896778 32519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:46.897028 32519 raft_consensus.cc:686] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [term 1 LEADER]: Becoming Leader. State: Replica: df063989ff284996a6709e94485bf4e0, State: Running, Role: LEADER
I20260929 18:44:46.897329 32519 consensus_queue.cc:237] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260929 18:44:46.898128 32509 sys_catalog.cc:564] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:46.898626 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:46.891807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55799 (local address 127.0.0.1:38491)
0929 18:44:46.891934 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:46.891939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:46.894298 (+ 2359us) server_negotiation.cc:407] Connection header received
0929 18:44:46.894357 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:46.894361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:46.894407 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:46.894484 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:46.894834 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:46.895595 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:46.896576 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:46.896867 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:46.896970 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:46.898087 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:46.898095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:46.898098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:46.898117 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:46.898149 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:46.898164 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:46.898257 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:46.898432 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 18:44:46.898488 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:44:46.899055 32521 sys_catalog.cc:455] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df063989ff284996a6709e94485bf4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } } }
I20260929 18:44:46.899358 32520 sys_catalog.cc:455] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df063989ff284996a6709e94485bf4e0. Latest consensus state: current_term: 1 leader_uuid: "df063989ff284996a6709e94485bf4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df063989ff284996a6709e94485bf4e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } } }
I20260929 18:44:46.899467 32521 sys_catalog.cc:458] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:46.899549 32520 sys_catalog.cc:458] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:46.900079 32528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:46.903719 32528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:46.911238 32528 catalog_manager.cc:1347] Generated new cluster ID: bb2f34f4ec6e439d802fae90071213c1
I20260929 18:44:46.911377 32528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:46.924561 32528 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:46.925604 32528 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:46.933588 32528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df063989ff284996a6709e94485bf4e0: Generated new TSK 0
I20260929 18:44:46.934530 32528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:46.946554 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32441
2026-09-29T18:44:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:47 dist-test-slave-cgv7 krb5kdc[32540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:47 dist-test-slave-cgv7 krb5kdc[32540](info): set up 2 sockets
Sep 29 18:44:47 dist-test-slave-cgv7 krb5kdc[32540](info): commencing operation
krb5kdc: starting...
W20260929 18:44:47.986555 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:48 dist-test-slave-cgv7 krb5kdc[32540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:48.118700 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:48.231716 32553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:48.232053 32553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:48.232131 32553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:48.236371 32553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:48.236481 32553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:48.236518 32553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:48.236557 32553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:48.236594 32553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:48.242051 32553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:48.243264 32553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:48.244360 32553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:48.253260 32558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:48.253271 32561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:48.253265 32559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:48.253435 32553 server_base.cc:1034] running on GCE node
I20260929 18:44:48.254177 32553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:48.255394 32553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:48.256585 32553 hybrid_clock.cc:648] HybridClock initialized: now 1790707488256554 us; error 45 us; skew 500 ppm
Sep 29 18:44:48 dist-test-slave-cgv7 krb5kdc[32540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707488, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:48.259783 32553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:48.260687 32553 webserver.cc:460] Webserver started at http://127.0.0.1:34735/ using document root <none> and password file <none>
I20260929 18:44:48.261227 32553 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:48.261291 32553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:48.261471 32553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:48.263808 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "c481436e965c4c6d8743331d7a3a0ab1"
format_stamp: "Formatted at 2026-09-29 18:44:48 on dist-test-slave-cgv7"
server_key: "6183e625076cd45c9fd2b387fb33c582"
server_key_iv: "862089fd79aa63ad275cfd33eb6419a2"
server_key_version: "encryptionkey@0"
I20260929 18:44:48.264359 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "c481436e965c4c6d8743331d7a3a0ab1"
format_stamp: "Formatted at 2026-09-29 18:44:48 on dist-test-slave-cgv7"
server_key: "6183e625076cd45c9fd2b387fb33c582"
server_key_iv: "862089fd79aa63ad275cfd33eb6419a2"
server_key_version: "encryptionkey@0"
I20260929 18:44:48.268160 32553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:44:48.270539 32568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:48.271776 32553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:48.271968 32553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "c481436e965c4c6d8743331d7a3a0ab1"
format_stamp: "Formatted at 2026-09-29 18:44:48 on dist-test-slave-cgv7"
server_key: "6183e625076cd45c9fd2b387fb33c582"
server_key_iv: "862089fd79aa63ad275cfd33eb6419a2"
server_key_version: "encryptionkey@0"
I20260929 18:44:48.272087 32553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:48.276855 32553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:48.280169 32553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:48.280375 32553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:48.287598 32553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39717
I20260929 18:44:48.287596 32620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39717 every 8 connection(s)
I20260929 18:44:48.288573 32553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:48.291445 32621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:48.295212 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32553
I20260929 18:44:48.295408 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:48.295670 11739 external_mini_cluster.cc:1428] Setting key 4ba9cc0f2d46fe76b5f899add119efa8
I20260929 18:44:48.297142 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1: Bootstrap starting.
I20260929 18:44:48.299531 32621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:48.300395 32621 log.cc:824] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:48.301007 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:48.297011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59111 (local address 127.0.0.1:39717)
0929 18:44:48.297484 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:48.297499 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:48.297533 (+ 34us) server_negotiation.cc:407] Connection header received
0929 18:44:48.298180 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:48.298196 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:48.298548 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:48.298834 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:48.299740 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:48.299745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:48.299757 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:48.299769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:48.299825 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:48.299849 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:48.299951 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:48.300101 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 18:44:48.300272 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":160,"threads_started":1}
I20260929 18:44:48.302368 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1: No bootstrap required, opened a new log
I20260929 18:44:48.304014 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:48.306615 32621 raft_consensus.cc:348] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260929 18:44:48.306833 32621 raft_consensus.cc:374] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:48.306887 32621 raft_consensus.cc:729] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c481436e965c4c6d8743331d7a3a0ab1, State: Initialized, Role: FOLLOWER
I20260929 18:44:48.307368 32621 consensus_queue.cc:260] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260929 18:44:48.307534 32621 raft_consensus.cc:388] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:48.307641 32621 raft_consensus.cc:482] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:48.307763 32621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:48.307926 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:48.306024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49431 (local address 127.0.0.1:39717)
0929 18:44:48.306135 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:48.306138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:48.306403 (+ 265us) server_negotiation.cc:407] Connection header received
0929 18:44:48.306555 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:48.306559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:48.306600 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:48.306683 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:48.307428 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:48.307431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:48.307433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:48.307445 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:48.307473 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:48.307495 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:48.307590 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:48.307698 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 18:44:48.307756 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 18:44:48.309033 32621 raft_consensus.cc:504] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260929 18:44:48.309373 32621 leader_election.cc:302] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c481436e965c4c6d8743331d7a3a0ab1; no voters:
I20260929 18:44:48.309682 32621 leader_election.cc:288] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:48.309801 32631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:48.309995 32631 raft_consensus.cc:686] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [term 1 LEADER]: Becoming Leader. State: Replica: c481436e965c4c6d8743331d7a3a0ab1, State: Running, Role: LEADER
I20260929 18:44:48.310417 32631 consensus_queue.cc:237] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260929 18:44:48.310948 32621 sys_catalog.cc:564] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:48.312199 32632 sys_catalog.cc:455] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c481436e965c4c6d8743331d7a3a0ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } } }
I20260929 18:44:48.312373 32632 sys_catalog.cc:458] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:48.312697 32633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c481436e965c4c6d8743331d7a3a0ab1. Latest consensus state: current_term: 1 leader_uuid: "c481436e965c4c6d8743331d7a3a0ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c481436e965c4c6d8743331d7a3a0ab1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } } }
I20260929 18:44:48.312801 32633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:48.313385 32641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:48.315979 32641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:48.321357 32641 catalog_manager.cc:1347] Generated new cluster ID: abd30e8567574566b06f0c1497060b02
I20260929 18:44:48.321442 32641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:48.346709 32641 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:48.347095 32641 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:48.357291 32641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c481436e965c4c6d8743331d7a3a0ab1: Generated new TSK 0
I20260929 18:44:48.358161 32641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:48.367199 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32553
2026-09-29T18:44:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:48 dist-test-slave-cgv7 krb5kdc[32652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:48 dist-test-slave-cgv7 krb5kdc[32652](info): set up 2 sockets
Sep 29 18:44:48 dist-test-slave-cgv7 krb5kdc[32652](info): commencing operation
krb5kdc: starting...
W20260929 18:44:49.398527 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:49.525872 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:49.638669 32665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:49.639015 32665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:49.639103 32665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:49.643198 32665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:49.643289 32665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:49.643327 32665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:49.643379 32665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:49.643417 32665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:49.648759 32665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:49.649881 32665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:49.650955 32665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:49.659251 32673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:49.659458 32665 server_base.cc:1034] running on GCE node
W20260929 18:44:49.659272 32671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:49.659593 32670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:49.660141 32665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:49.661239 32665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:49.662390 32665 hybrid_clock.cc:648] HybridClock initialized: now 1790707489662362 us; error 42 us; skew 500 ppm
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707489, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:49.665215 32665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:49.666406 32665 webserver.cc:460] Webserver started at http://127.0.0.1:46773/ using document root <none> and password file <none>
I20260929 18:44:49.666985 32665 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:49.667078 32665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:49.667315 32665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:49.670145 32665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "58f36273d44f4ac18cdee1962d8fae66"
format_stamp: "Formatted at 2026-09-29 18:44:49 on dist-test-slave-cgv7"
server_key: "c4c1d2da36fc0831d897d0cbb6e9b889"
server_key_iv: "cbe7cca5c5ea52c9c370af9e57ad523a"
server_key_version: "encryptionkey@0"
I20260929 18:44:49.670926 32665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "58f36273d44f4ac18cdee1962d8fae66"
format_stamp: "Formatted at 2026-09-29 18:44:49 on dist-test-slave-cgv7"
server_key: "c4c1d2da36fc0831d897d0cbb6e9b889"
server_key_iv: "cbe7cca5c5ea52c9c370af9e57ad523a"
server_key_version: "encryptionkey@0"
I20260929 18:44:49.675997 32665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 18:44:49.679067 32680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:49.680299 32665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260929 18:44:49.680420 32665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "58f36273d44f4ac18cdee1962d8fae66"
format_stamp: "Formatted at 2026-09-29 18:44:49 on dist-test-slave-cgv7"
server_key: "c4c1d2da36fc0831d897d0cbb6e9b889"
server_key_iv: "cbe7cca5c5ea52c9c370af9e57ad523a"
server_key_version: "encryptionkey@0"
I20260929 18:44:49.680512 32665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:49.695771 32665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:49.698679 32665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:49.698841 32665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:49.705914 32665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36391
I20260929 18:44:49.705942 32732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36391 every 8 connection(s)
I20260929 18:44:49.706897 32665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:49.709481 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:49.712472 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 32665
I20260929 18:44:49.712627 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:49.712965 11739 external_mini_cluster.cc:1428] Setting key eeebf8f01cd6221bf2bdfae19cc392a3
I20260929 18:44:49.715775 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66: Bootstrap starting.
I20260929 18:44:49.718456 32733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:49.719292 32733 log.cc:824] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:49.721531 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66: No bootstrap required, opened a new log
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 18:44:49.725365 32733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260929 18:44:49.725601 32733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:49.725659 32733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f36273d44f4ac18cdee1962d8fae66, State: Initialized, Role: FOLLOWER
I20260929 18:44:49.726135 32733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260929 18:44:49.726332 32733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:49.726416 32733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:49.726522 32733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:49.727646 32733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260929 18:44:49.728067 32733 leader_election.cc:302] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58f36273d44f4ac18cdee1962d8fae66; no voters:
I20260929 18:44:49.728463 32733 leader_election.cc:288] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:49.728579 32738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:49.728829 32738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [term 1 LEADER]: Becoming Leader. State: Replica: 58f36273d44f4ac18cdee1962d8fae66, State: Running, Role: LEADER
I20260929 18:44:49.729583 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:49.714504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56835 (local address 127.0.0.1:36391)
0929 18:44:49.714941 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:49.714960 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:49.715001 (+ 41us) server_negotiation.cc:407] Connection header received
0929 18:44:49.715786 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:49.715811 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:49.716231 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:49.716588 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:49.717320 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:49.718830 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:49.719874 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:49.720520 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:49.720671 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:49.723586 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:49.723591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:49.723606 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:49.723633 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:49.726081 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:49.726738 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:49.726743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:49.726748 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:49.726869 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:49.727232 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:49.727234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:49.727236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:49.727651 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:49.727914 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:49.728156 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 18:44:49.728399 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":129,"threads_started":1}
I20260929 18:44:49.729815 32733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:49.729913 32738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260929 18:44:49.731685 32739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58f36273d44f4ac18cdee1962d8fae66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260929 18:44:49.731827 32739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:49.732206 32740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f36273d44f4ac18cdee1962d8fae66. Latest consensus state: current_term: 1 leader_uuid: "58f36273d44f4ac18cdee1962d8fae66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f36273d44f4ac18cdee1962d8fae66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260929 18:44:49.732566 32752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:49.732589 32740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:49.732995 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:49.735545 32752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:49.741264 32752 catalog_manager.cc:1347] Generated new cluster ID: b263576242874e62a8186098e82ef5a8
I20260929 18:44:49.741406 32752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:49.747184 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:49.735260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:53283 (local address 127.0.0.1:36391)
0929 18:44:49.735368 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:49.735373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:49.738622 (+ 3249us) server_negotiation.cc:407] Connection header received
0929 18:44:49.738748 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:49.738754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:49.738814 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:49.738924 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:49.739822 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:49.740654 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:49.741789 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:49.742043 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:49.742160 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:49.743674 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:49.743680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:49.743688 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:49.743712 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:49.745192 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:49.745697 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:49.745702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:49.745709 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:49.745780 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:49.746156 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:49.746159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:49.746163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:49.746419 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:49.746597 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:49.746905 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 18:44:49.747007 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 18:44:49.750952 32752 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:49.751919 32752 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:49.764766 32752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58f36273d44f4ac18cdee1962d8fae66: Generated new TSK 0
I20260929 18:44:49.765609 32752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:49.771394 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 32665
W20260929 18:44:49.776095 32650 connection.cc:535] client connection to 127.0.0.1:36391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36391): Connection reset by peer (error 104)
2026-09-29T18:44:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32764](info): set up 2 sockets
Sep 29 18:44:49 dist-test-slave-cgv7 krb5kdc[32764](info): commencing operation
krb5kdc: starting...
W20260929 18:44:50.810487 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:50 dist-test-slave-cgv7 krb5kdc[32764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:50.935503 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:51.047902 309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:51.048179 309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:51.048223 309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:51.052500 309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:51.052558 309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:51.052577 309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:51.052594 309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:51.052650 309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:51.057837 309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:51.058920 309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:51.059917 309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:51.068463 317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:51.068463 315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:51.068496 314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:51.068670 309 server_base.cc:1034] running on GCE node
I20260929 18:44:51.069329 309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:51.070585 309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:51.071790 309 hybrid_clock.cc:648] HybridClock initialized: now 1790707491071736 us; error 68 us; skew 500 ppm
Sep 29 18:44:51 dist-test-slave-cgv7 krb5kdc[32764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707491, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:51.079030 309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:51.080062 309 webserver.cc:460] Webserver started at http://127.0.0.1:43627/ using document root <none> and password file <none>
I20260929 18:44:51.080669 309 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:51.080766 309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:51.081007 309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:51.083300 309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "23e4349d204a47ccb7fc2a781b7f0cf0"
format_stamp: "Formatted at 2026-09-29 18:44:51 on dist-test-slave-cgv7"
server_key: "4c43d2c80153f08c1f569c5da3a67807"
server_key_iv: "057ba93911c4f18ba3c439bcf50ded84"
server_key_version: "encryptionkey@0"
I20260929 18:44:51.083961 309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "23e4349d204a47ccb7fc2a781b7f0cf0"
format_stamp: "Formatted at 2026-09-29 18:44:51 on dist-test-slave-cgv7"
server_key: "4c43d2c80153f08c1f569c5da3a67807"
server_key_iv: "057ba93911c4f18ba3c439bcf50ded84"
server_key_version: "encryptionkey@0"
I20260929 18:44:51.087823 309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:44:51.090377 324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:51.091650 309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:44:51.091784 309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "23e4349d204a47ccb7fc2a781b7f0cf0"
format_stamp: "Formatted at 2026-09-29 18:44:51 on dist-test-slave-cgv7"
server_key: "4c43d2c80153f08c1f569c5da3a67807"
server_key_iv: "057ba93911c4f18ba3c439bcf50ded84"
server_key_version: "encryptionkey@0"
I20260929 18:44:51.091898 309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:51.110560 309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:51.113957 309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:51.114171 309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:51.121589 309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39201
I20260929 18:44:51.121604 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39201 every 8 connection(s)
I20260929 18:44:51.122730 309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:51.125655 377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:51.131506 377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0: Bootstrap starting.
I20260929 18:44:51.132267 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 309
I20260929 18:44:51.132444 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:51.132730 11739 external_mini_cluster.cc:1428] Setting key 6669f8e22b79daa6357cb677898c522d
I20260929 18:44:51.134492 377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:51.135247 377 log.cc:824] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:51.137207 377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0: No bootstrap required, opened a new log
I20260929 18:44:51.140777 377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260929 18:44:51.141086 377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:51.141166 377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e4349d204a47ccb7fc2a781b7f0cf0, State: Initialized, Role: FOLLOWER
I20260929 18:44:51.141613 377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260929 18:44:51.141762 377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:51.141795 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:51.133923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:47509 (local address 127.0.0.1:39201)
0929 18:44:51.134379 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:51.134395 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:51.134433 (+ 38us) server_negotiation.cc:407] Connection header received
0929 18:44:51.135140 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:51.135157 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:51.135570 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:51.135941 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:51.136364 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:51.137377 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:51.138292 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:51.138795 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:51.138977 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:51.139908 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:51.139913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:51.139926 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:51.140313 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:51.140368 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:51.140390 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:51.140531 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:51.140709 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 18:44:51.140863 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":141,"threads_started":1}
I20260929 18:44:51.141846 377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:51.141957 377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:51.143112 377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260929 18:44:51.143471 377 leader_election.cc:302] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23e4349d204a47ccb7fc2a781b7f0cf0; no voters:
I20260929 18:44:51.143791 377 leader_election.cc:288] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:51.144160 382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:51.144454 382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [term 1 LEADER]: Becoming Leader. State: Replica: 23e4349d204a47ccb7fc2a781b7f0cf0, State: Running, Role: LEADER
I20260929 18:44:51.144806 382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260929 18:44:51.144955 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:51.145148 377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:51.145915 382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23e4349d204a47ccb7fc2a781b7f0cf0. Latest consensus state: current_term: 1 leader_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } } }
I20260929 18:44:51.146085 382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:51.146521 392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:51.147480 383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e4349d204a47ccb7fc2a781b7f0cf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } } }
I20260929 18:44:51.147591 383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:51.150115 392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:51.153236 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:51.147969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:55941 (local address 127.0.0.1:39201)
0929 18:44:51.148101 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:51.148106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:51.148359 (+ 253us) server_negotiation.cc:407] Connection header received
0929 18:44:51.148490 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:51.148493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:51.148552 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:51.148633 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:51.148917 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:51.149701 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:51.151273 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:51.151645 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:51.151765 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:51.152732 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:51.152736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:51.152740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:51.152754 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:51.152783 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:51.152797 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:51.152893 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:51.153060 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 18:44:51.153097 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:44:51.156178 392 catalog_manager.cc:1347] Generated new cluster ID: ed29156f3716450d89d4c121732ae4a3
I20260929 18:44:51.156243 392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:51.169198 392 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:51.170526 392 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:51.180212 392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23e4349d204a47ccb7fc2a781b7f0cf0: Generated new TSK 0
I20260929 18:44:51.181102 392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:51.189909 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 309
W20260929 18:44:51.195101 32762 connection.cc:535] client connection to 127.0.0.1:39201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39201): Connection reset by peer (error 104)
2026-09-29T18:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:51 dist-test-slave-cgv7 krb5kdc[408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:51 dist-test-slave-cgv7 krb5kdc[408](info): set up 2 sockets
Sep 29 18:44:51 dist-test-slave-cgv7 krb5kdc[408](info): commencing operation
krb5kdc: starting...
W20260929 18:44:52.230603 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:52 dist-test-slave-cgv7 krb5kdc[408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:52.361116 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:52.477830 421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:52.478106 421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:52.478150 421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:52.482251 421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:52.482357 421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:52.482376 421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:52.482393 421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:52.482447 421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:52.488025 421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:52.489115 421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:52.490229 421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:44:52.498795 421 server_base.cc:1034] running on GCE node
W20260929 18:44:52.498745 426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:52.498764 427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:52.498745 429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:52.499575 421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:52.500691 421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:52.501852 421 hybrid_clock.cc:648] HybridClock initialized: now 1790707492501826 us; error 43 us; skew 500 ppm
Sep 29 18:44:52 dist-test-slave-cgv7 krb5kdc[408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707492, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:52.504747 421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:52.505710 421 webserver.cc:460] Webserver started at http://127.0.0.1:37669/ using document root <none> and password file <none>
I20260929 18:44:52.506361 421 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:52.506460 421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:52.506688 421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:52.508890 421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "bbe043b94aa14c4182f5c79fd59f6eec"
format_stamp: "Formatted at 2026-09-29 18:44:52 on dist-test-slave-cgv7"
server_key: "1900937b01cbd53407f7b886cddf7da8"
server_key_iv: "6f949d12d9fe0dc7257d0ad2cec404ce"
server_key_version: "encryptionkey@0"
I20260929 18:44:52.509497 421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "bbe043b94aa14c4182f5c79fd59f6eec"
format_stamp: "Formatted at 2026-09-29 18:44:52 on dist-test-slave-cgv7"
server_key: "1900937b01cbd53407f7b886cddf7da8"
server_key_iv: "6f949d12d9fe0dc7257d0ad2cec404ce"
server_key_version: "encryptionkey@0"
I20260929 18:44:52.513454 421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:52.516069 436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:52.517284 421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:44:52.517441 421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "bbe043b94aa14c4182f5c79fd59f6eec"
format_stamp: "Formatted at 2026-09-29 18:44:52 on dist-test-slave-cgv7"
server_key: "1900937b01cbd53407f7b886cddf7da8"
server_key_iv: "6f949d12d9fe0dc7257d0ad2cec404ce"
server_key_version: "encryptionkey@0"
I20260929 18:44:52.517567 421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:52.522413 421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:52.525732 421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:52.525939 421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:52.533165 421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35647
I20260929 18:44:52.533176 488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35647 every 8 connection(s)
I20260929 18:44:52.534117 421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:52.537010 489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:52.537968 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 421
I20260929 18:44:52.538141 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:52.538476 11739 external_mini_cluster.cc:1428] Setting key 332ab9512be1ff1e2ddd92ace7f55782
I20260929 18:44:52.543720 489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec: Bootstrap starting.
I20260929 18:44:52.543916 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:52.539918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:46283 (local address 127.0.0.1:35647)
0929 18:44:52.540389 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:52.540404 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:52.540441 (+ 37us) server_negotiation.cc:407] Connection header received
0929 18:44:52.541079 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:52.541094 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:52.541391 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:52.541740 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:52.542658 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:52.542664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:52.542676 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:52.542686 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:52.542741 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:52.542760 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:52.542913 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:52.542985 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 18:44:52.543156 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260929 18:44:52.546458 489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:52.546903 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:52.547323 489 log.cc:824] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:52.549494 489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec: No bootstrap required, opened a new log
I20260929 18:44:52.550773 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:52.549009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:45633 (local address 127.0.0.1:35647)
0929 18:44:52.549142 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:52.549147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:52.549266 (+ 119us) server_negotiation.cc:407] Connection header received
0929 18:44:52.549351 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:52.549354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:52.549387 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:52.549473 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:52.550316 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:52.550319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:52.550322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:52.550334 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:52.550360 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:52.550374 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:52.550464 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:52.550624 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 18:44:52.550664 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:44:52.553632 489 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } }
I20260929 18:44:52.553864 489 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:52.553898 489 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe043b94aa14c4182f5c79fd59f6eec, State: Initialized, Role: FOLLOWER
I20260929 18:44:52.554262 489 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } }
I20260929 18:44:52.554466 489 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:52.554514 489 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:52.554605 489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:52.555521 489 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } }
I20260929 18:44:52.555819 489 leader_election.cc:302] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbe043b94aa14c4182f5c79fd59f6eec; no voters:
I20260929 18:44:52.556165 489 leader_election.cc:288] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:52.556245 499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:52.556469 499 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [term 1 LEADER]: Becoming Leader. State: Replica: bbe043b94aa14c4182f5c79fd59f6eec, State: Running, Role: LEADER
I20260929 18:44:52.556856 499 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } }
I20260929 18:44:52.557613 489 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:52.558736 500 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbe043b94aa14c4182f5c79fd59f6eec. Latest consensus state: current_term: 1 leader_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } } }
I20260929 18:44:52.558749 501 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe043b94aa14c4182f5c79fd59f6eec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35647 } } }
I20260929 18:44:52.558892 500 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:52.559211 501 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:52.559422 508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:52.562863 508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:52.569278 508 catalog_manager.cc:1347] Generated new cluster ID: df143c9228f4492fb8254c1435c93da7
I20260929 18:44:52.569370 508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:52.581987 508 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:52.582391 508 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:52.590785 508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbe043b94aa14c4182f5c79fd59f6eec: Generated new TSK 0
I20260929 18:44:52.591904 508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:52.600775 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 421
2026-09-29T18:44:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:52 dist-test-slave-cgv7 krb5kdc[520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:52 dist-test-slave-cgv7 krb5kdc[520](info): set up 2 sockets
Sep 29 18:44:52 dist-test-slave-cgv7 krb5kdc[520](info): commencing operation
krb5kdc: starting...
W20260929 18:44:53.654457 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:53 dist-test-slave-cgv7 krb5kdc[520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:53.781811 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:53.894732 533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:53.895035 533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:53.895079 533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:53.899654 533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:53.899719 533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:53.899739 533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:53.899756 533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:53.899772 533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:53.905220 533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:53.906291 533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:53.907385 533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:53.916464 541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:53.916429 538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:53.916567 533 server_base.cc:1034] running on GCE node
W20260929 18:44:53.916389 539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:53.917279 533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:53.918478 533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:53.919672 533 hybrid_clock.cc:648] HybridClock initialized: now 1790707493919652 us; error 34 us; skew 500 ppm
Sep 29 18:44:53 dist-test-slave-cgv7 krb5kdc[520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707493, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:53.922580 533 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:53.923564 533 webserver.cc:460] Webserver started at http://127.0.0.1:36703/ using document root <none> and password file <none>
I20260929 18:44:53.924146 533 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:53.924239 533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:53.924479 533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:53.926630 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "d99e1adf9f054f2383844c6bde6978d0"
format_stamp: "Formatted at 2026-09-29 18:44:53 on dist-test-slave-cgv7"
server_key: "becfd5c96be3aa6e0463f87726bc224a"
server_key_iv: "eeabb07a91167d956690b5e8aa453e68"
server_key_version: "encryptionkey@0"
I20260929 18:44:53.927214 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "d99e1adf9f054f2383844c6bde6978d0"
format_stamp: "Formatted at 2026-09-29 18:44:53 on dist-test-slave-cgv7"
server_key: "becfd5c96be3aa6e0463f87726bc224a"
server_key_iv: "eeabb07a91167d956690b5e8aa453e68"
server_key_version: "encryptionkey@0"
I20260929 18:44:53.931103 533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 18:44:53.933543 548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:53.934886 533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:44:53.935030 533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "d99e1adf9f054f2383844c6bde6978d0"
format_stamp: "Formatted at 2026-09-29 18:44:53 on dist-test-slave-cgv7"
server_key: "becfd5c96be3aa6e0463f87726bc224a"
server_key_iv: "eeabb07a91167d956690b5e8aa453e68"
server_key_version: "encryptionkey@0"
I20260929 18:44:53.935150 533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:53.968633 533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:53.971863 533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:53.972081 533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:53.979362 533 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45025
I20260929 18:44:53.979388 600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45025 every 8 connection(s)
I20260929 18:44:53.980363 533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:53.983175 601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:53.988560 601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: Bootstrap starting.
I20260929 18:44:53.988756 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 533
I20260929 18:44:53.988890 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:53.989172 11739 external_mini_cluster.cc:1428] Setting key 94e5ffe341c980442e49d25d0c960860
I20260929 18:44:53.991242 601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:53.991956 601 log.cc:824] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:53.994128 601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: No bootstrap required, opened a new log
I20260929 18:44:53.998098 601 raft_consensus.cc:348] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260929 18:44:53.998394 601 raft_consensus.cc:374] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:53.998469 601 raft_consensus.cc:729] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99e1adf9f054f2383844c6bde6978d0, State: Initialized, Role: FOLLOWER
Sep 29 18:44:53 dist-test-slave-cgv7 krb5kdc[520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 18:44:53.998955 601 consensus_queue.cc:260] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260929 18:44:53.999078 601 raft_consensus.cc:388] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:53.999159 601 raft_consensus.cc:482] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:53.999260 601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:54.000528 601 raft_consensus.cc:504] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260929 18:44:54.000936 601 leader_election.cc:302] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d99e1adf9f054f2383844c6bde6978d0; no voters:
I20260929 18:44:54.001299 601 leader_election.cc:288] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:54.001427 606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:54.001681 606 raft_consensus.cc:686] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [term 1 LEADER]: Becoming Leader. State: Replica: d99e1adf9f054f2383844c6bde6978d0, State: Running, Role: LEADER
I20260929 18:44:54.002144 606 consensus_queue.cc:237] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260929 18:44:54.002689 601 sys_catalog.cc:564] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:54.004946 607 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d99e1adf9f054f2383844c6bde6978d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } } }
I20260929 18:44:54.005518 607 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:54.005342 608 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d99e1adf9f054f2383844c6bde6978d0. Latest consensus state: current_term: 1 leader_uuid: "d99e1adf9f054f2383844c6bde6978d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99e1adf9f054f2383844c6bde6978d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } } }
I20260929 18:44:54.005631 608 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:54.005245 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:53.990594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:55377 (local address 127.0.0.1:45025)
0929 18:44:53.991010 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:53.991030 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:53.991066 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:44:53.991764 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:53.991782 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:53.992176 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:53.992466 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:53.993243 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:53.994731 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:53.995630 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:53.996344 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:53.996501 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:53.999359 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:53.999366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:53.999384 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:53.999418 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:54.001454 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:54.002132 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:54.002138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:54.002144 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:54.002304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:54.002682 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:54.002686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:54.002687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:54.003253 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:54.003482 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:54.004000 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 18:44:54.004229 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":139,"threads_started":1}
W20260929 18:44:54.007892 620 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:44:54.007967 620 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:44:54.008044 622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:54.008944 622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:54.008953 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:54.015245 622 catalog_manager.cc:1347] Generated new cluster ID: 48dd43546c004c40a1188bdba4545454
I20260929 18:44:54.015319 622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:54.019627 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:54.010979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:35251 (local address 127.0.0.1:45025)
0929 18:44:54.011089 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:54.011093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:54.011279 (+ 186us) server_negotiation.cc:407] Connection header received
0929 18:44:54.011438 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:54.011441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:54.011492 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:54.011573 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:54.012224 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:54.012839 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:54.014061 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:54.014411 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:54.014484 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:54.016115 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:54.016118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:54.016121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:54.016144 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:54.017656 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:54.018161 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:54.018164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:54.018166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:54.018247 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:54.018661 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:54.018664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:54.018666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:54.018883 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:54.019067 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:54.019322 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 18:44:54.019425 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 18:44:54.039790 622 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:54.040869 622 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:54.048663 622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d99e1adf9f054f2383844c6bde6978d0: Generated new TSK 0
I20260929 18:44:54.049417 622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:54.058032 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 533
W20260929 18:44:54.063002 518 connection.cc:535] client connection to 127.0.0.1:45025 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45025): Connection reset by peer (error 104)
2026-09-29T18:44:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:54 dist-test-slave-cgv7 krb5kdc[632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:54 dist-test-slave-cgv7 krb5kdc[632](info): set up 2 sockets
Sep 29 18:44:54 dist-test-slave-cgv7 krb5kdc[632](info): commencing operation
krb5kdc: starting...
W20260929 18:44:55.106705 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:55 dist-test-slave-cgv7 krb5kdc[632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:55.233455 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:55.343230 645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:55.343521 645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:55.343564 645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:55.347519 645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:55.347576 645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:55.347595 645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:55.347612 645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:55.347626 645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:55.353443 645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43111
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:55.354516 645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:55.355538 645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:55.364154 653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:55.364168 650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:55.364168 651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:55.364400 645 server_base.cc:1034] running on GCE node
I20260929 18:44:55.365053 645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:55.366199 645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:55.367405 645 hybrid_clock.cc:648] HybridClock initialized: now 1790707495367394 us; error 25 us; skew 500 ppm
Sep 29 18:44:55 dist-test-slave-cgv7 krb5kdc[632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707495, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:55.370250 645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:55.371236 645 webserver.cc:460] Webserver started at http://127.0.0.1:43723/ using document root <none> and password file <none>
I20260929 18:44:55.371834 645 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:55.371932 645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:55.372174 645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:55.374410 645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "3c3e4074e98147828c2fad6c1e2933f5"
format_stamp: "Formatted at 2026-09-29 18:44:55 on dist-test-slave-cgv7"
server_key: "de42f7cf1bcb5f63c323e5024b7dc946"
server_key_iv: "c8ae9218ac64cf0a767798220b040f72"
server_key_version: "encryptionkey@0"
I20260929 18:44:55.375011 645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "3c3e4074e98147828c2fad6c1e2933f5"
format_stamp: "Formatted at 2026-09-29 18:44:55 on dist-test-slave-cgv7"
server_key: "de42f7cf1bcb5f63c323e5024b7dc946"
server_key_iv: "c8ae9218ac64cf0a767798220b040f72"
server_key_version: "encryptionkey@0"
I20260929 18:44:55.379274 645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:44:55.382128 660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:55.383543 645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 18:44:55.383698 645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "3c3e4074e98147828c2fad6c1e2933f5"
format_stamp: "Formatted at 2026-09-29 18:44:55 on dist-test-slave-cgv7"
server_key: "de42f7cf1bcb5f63c323e5024b7dc946"
server_key_iv: "c8ae9218ac64cf0a767798220b040f72"
server_key_version: "encryptionkey@0"
I20260929 18:44:55.383827 645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:55.409015 645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:55.412281 645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:55.412503 645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:55.419425 645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43111
I20260929 18:44:55.419450 712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43111 every 8 connection(s)
I20260929 18:44:55.420387 645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:55.423223 713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:55.429016 713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5: Bootstrap starting.
I20260929 18:44:55.430044 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 645
I20260929 18:44:55.430210 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:55.430546 11739 external_mini_cluster.cc:1428] Setting key f468dde531e17549e909cf286157e36c
I20260929 18:44:55.431952 713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:55.432696 713 log.cc:824] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:55.434868 713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5: No bootstrap required, opened a new log
I20260929 18:44:55.439257 713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } }
I20260929 18:44:55.439512 713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:55.439599 713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3e4074e98147828c2fad6c1e2933f5, State: Initialized, Role: FOLLOWER
I20260929 18:44:55.440044 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:55.431723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:37853 (local address 127.0.0.1:43111)
0929 18:44:55.432214 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:55.432231 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:55.432265 (+ 34us) server_negotiation.cc:407] Connection header received
0929 18:44:55.432990 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:55.433009 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:55.433479 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:55.433785 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:55.434108 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:55.435185 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:55.436361 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:55.436839 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:55.436982 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:55.437933 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:55.437938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:55.437952 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:55.438564 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:55.438636 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:55.438660 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:55.438774 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:55.438985 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 18:44:55.439156 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":173,"threads_started":1}
I20260929 18:44:55.440090 713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } }
I20260929 18:44:55.440311 713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:55.440388 713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:55.440493 713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:55.441583 713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } }
I20260929 18:44:55.441964 713 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c3e4074e98147828c2fad6c1e2933f5; no voters:
I20260929 18:44:55.442332 713 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:55.442402 718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:55.442595 718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3e4074e98147828c2fad6c1e2933f5, State: Running, Role: LEADER
I20260929 18:44:55.442945 718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } }
I20260929 18:44:55.443210 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:55.443629 713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:55.444653 720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c3e4074e98147828c2fad6c1e2933f5. Latest consensus state: current_term: 1 leader_uuid: "3c3e4074e98147828c2fad6c1e2933f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } } }
I20260929 18:44:55.444782 720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:55.445113 719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c3e4074e98147828c2fad6c1e2933f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3e4074e98147828c2fad6c1e2933f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43111 } } }
I20260929 18:44:55.445261 719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:55.445561 731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:55.449606 731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:55.450541 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:55.446390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:60463 (local address 127.0.0.1:43111)
0929 18:44:55.446516 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:55.446521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:55.446877 (+ 356us) server_negotiation.cc:407] Connection header received
0929 18:44:55.446989 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:55.446991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:55.447032 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:55.447097 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:55.447411 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:55.447907 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:55.448747 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:55.448981 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:55.449099 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:55.450022 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:55.450025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:55.450028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:55.450042 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:55.450072 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:55.450083 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:55.450182 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:55.450347 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 18:44:55.450379 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:44:55.455772 731 catalog_manager.cc:1347] Generated new cluster ID: 784c9868817043cc89598629d8cad3ac
I20260929 18:44:55.455878 731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:55.462884 731 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:55.463830 731 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:55.486229 731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c3e4074e98147828c2fad6c1e2933f5: Generated new TSK 0
I20260929 18:44:55.487181 731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:55.498513 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 645
W20260929 18:44:55.503857 630 connection.cc:535] client connection to 127.0.0.1:43111 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43111): Connection reset by peer (error 104)
2026-09-29T18:44:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:55 dist-test-slave-cgv7 krb5kdc[744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:55 dist-test-slave-cgv7 krb5kdc[744](info): set up 2 sockets
Sep 29 18:44:55 dist-test-slave-cgv7 krb5kdc[744](info): commencing operation
krb5kdc: starting...
W20260929 18:44:56.546531 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:56 dist-test-slave-cgv7 krb5kdc[744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 18:44:56.673717 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:56.785586 757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:56.785924 757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:56.786000 757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:56.790073 757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:56.790187 757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:56.790226 757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:56.790292 757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:56.790333 757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:56.795804 757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:56.796944 757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:56.797987 757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:56.806375 763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:56.806376 765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:56.806380 762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:56.806972 757 server_base.cc:1034] running on GCE node
I20260929 18:44:56.807504 757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:56.808604 757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:56.809782 757 hybrid_clock.cc:648] HybridClock initialized: now 1790707496809765 us; error 30 us; skew 500 ppm
Sep 29 18:44:56 dist-test-slave-cgv7 krb5kdc[744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:56.816537 757 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 18:44:56.817553 757 webserver.cc:460] Webserver started at http://127.0.0.1:38983/ using document root <none> and password file <none>
I20260929 18:44:56.818179 757 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:56.818328 757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:56.818533 757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:56.820664 757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "53c1101dcf2e45b2a0330f94d0db3c3c"
format_stamp: "Formatted at 2026-09-29 18:44:56 on dist-test-slave-cgv7"
server_key: "d854a6a7fc46722f2a8380bcc2e858ee"
server_key_iv: "295cad9cbb2969722708c9e981f3e0e8"
server_key_version: "encryptionkey@0"
I20260929 18:44:56.821206 757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "53c1101dcf2e45b2a0330f94d0db3c3c"
format_stamp: "Formatted at 2026-09-29 18:44:56 on dist-test-slave-cgv7"
server_key: "d854a6a7fc46722f2a8380bcc2e858ee"
server_key_iv: "295cad9cbb2969722708c9e981f3e0e8"
server_key_version: "encryptionkey@0"
I20260929 18:44:56.825034 757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 18:44:56.827591 772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:56.828954 757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:44:56.829104 757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "53c1101dcf2e45b2a0330f94d0db3c3c"
format_stamp: "Formatted at 2026-09-29 18:44:56 on dist-test-slave-cgv7"
server_key: "d854a6a7fc46722f2a8380bcc2e858ee"
server_key_iv: "295cad9cbb2969722708c9e981f3e0e8"
server_key_version: "encryptionkey@0"
I20260929 18:44:56.829217 757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:56.833916 757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:56.837335 757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:56.837540 757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:56.844556 757 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43497
I20260929 18:44:56.844609 824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43497 every 8 connection(s)
I20260929 18:44:56.845535 757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:56.848512 825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:56.850318 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 757
I20260929 18:44:56.850454 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:56.850751 11739 external_mini_cluster.cc:1428] Setting key f27e8c8dd66c580500a9aa96e8c272c4
I20260929 18:44:56.855688 825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c: Bootstrap starting.
I20260929 18:44:56.856594 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:56.852228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:43453 (local address 127.0.0.1:43497)
0929 18:44:56.852666 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:56.852685 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:56.852727 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:44:56.853631 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:56.853657 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:56.854055 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:56.854469 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:56.855197 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:56.855203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:56.855218 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:56.855230 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:56.855292 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:56.855318 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:56.855460 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:56.855526 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 18:44:56.855683 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":116,"threads_started":1}
I20260929 18:44:56.858444 825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:56.859329 825 log.cc:824] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:56.859889 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:56.861337 825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c: No bootstrap required, opened a new log
I20260929 18:44:56.863607 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:56.861955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56681 (local address 127.0.0.1:43497)
0929 18:44:56.862076 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:56.862081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:56.862221 (+ 140us) server_negotiation.cc:407] Connection header received
0929 18:44:56.862386 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:56.862390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:56.862425 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:56.862509 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:56.863240 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:56.863244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:56.863247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:56.863258 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:56.863283 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:56.863296 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:56.863386 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:56.863450 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 18:44:56.863489 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 18:44:56.865350 825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260929 18:44:56.865576 825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:56.865624 825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c1101dcf2e45b2a0330f94d0db3c3c, State: Initialized, Role: FOLLOWER
I20260929 18:44:56.866046 825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260929 18:44:56.866196 825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:56.866297 825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:56.866425 825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:56.867326 825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260929 18:44:56.867668 825 leader_election.cc:302] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53c1101dcf2e45b2a0330f94d0db3c3c; no voters:
I20260929 18:44:56.868000 825 leader_election.cc:288] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:56.868067 835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:56.868316 835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [term 1 LEADER]: Becoming Leader. State: Replica: 53c1101dcf2e45b2a0330f94d0db3c3c, State: Running, Role: LEADER
I20260929 18:44:56.868666 835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260929 18:44:56.869240 825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:56.870301 836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } } }
I20260929 18:44:56.870431 836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:56.870808 844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:56.870750 837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53c1101dcf2e45b2a0330f94d0db3c3c. Latest consensus state: current_term: 1 leader_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c1101dcf2e45b2a0330f94d0db3c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } } }
I20260929 18:44:56.871001 837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:56.873991 844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:56.880082 844 catalog_manager.cc:1347] Generated new cluster ID: 629e49c5f4714e4ea5940bb4f9b14e0a
I20260929 18:44:56.880156 844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:56.903455 844 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:56.903919 844 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:56.909789 844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53c1101dcf2e45b2a0330f94d0db3c3c: Generated new TSK 0
I20260929 18:44:56.910691 844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:56.915860 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 757
2026-09-29T18:44:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:56 dist-test-slave-cgv7 krb5kdc[856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:56 dist-test-slave-cgv7 krb5kdc[856](info): set up 2 sockets
Sep 29 18:44:56 dist-test-slave-cgv7 krb5kdc[856](info): commencing operation
krb5kdc: starting...
W20260929 18:44:57.962636 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:44:58.090042 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39089
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:39089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:58.201213 869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:58.201504 869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:58.201578 869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:58.205708 869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:58.205777 869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:58.205797 869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:58.205854 869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:58.205891 869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:58.211380 869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:39089
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:58.212523 869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:58.213567 869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:58.222100 875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:58.222111 877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:58.222169 869 server_base.cc:1034] running on GCE node
W20260929 18:44:58.222110 874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:58.222959 869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:58.224094 869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:58.225281 869 hybrid_clock.cc:648] HybridClock initialized: now 1790707498225265 us; error 31 us; skew 500 ppm
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707498, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:58.228171 869 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:58.229079 869 webserver.cc:460] Webserver started at http://127.11.118.254:37659/ using document root <none> and password file <none>
I20260929 18:44:58.229673 869 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:58.229775 869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:58.230013 869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:58.232244 869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "1efb864ee8b54ed4842b1a7adee88c96"
format_stamp: "Formatted at 2026-09-29 18:44:58 on dist-test-slave-cgv7"
server_key: "e7e476a6e93b48e57865a3e2d40828f3"
server_key_iv: "1028d718463b710187bf6b876730e314"
server_key_version: "encryptionkey@0"
I20260929 18:44:58.232810 869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "1efb864ee8b54ed4842b1a7adee88c96"
format_stamp: "Formatted at 2026-09-29 18:44:58 on dist-test-slave-cgv7"
server_key: "e7e476a6e93b48e57865a3e2d40828f3"
server_key_iv: "1028d718463b710187bf6b876730e314"
server_key_version: "encryptionkey@0"
I20260929 18:44:58.236789 869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:58.239307 884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:58.240566 869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:44:58.240702 869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "1efb864ee8b54ed4842b1a7adee88c96"
format_stamp: "Formatted at 2026-09-29 18:44:58 on dist-test-slave-cgv7"
server_key: "e7e476a6e93b48e57865a3e2d40828f3"
server_key_iv: "1028d718463b710187bf6b876730e314"
server_key_version: "encryptionkey@0"
I20260929 18:44:58.240816 869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:58.256963 869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:58.260514 869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:58.260727 869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:58.268114 869 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:39089
I20260929 18:44:58.268170 936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:39089 every 8 connection(s)
I20260929 18:44:58.269230 869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:58.272194 937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:58.276522 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 869
I20260929 18:44:58.276650 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:58.276947 11739 external_mini_cluster.cc:1428] Setting key cdce5c8cc31162cf524f89c8fe2202d9
I20260929 18:44:58.278234 937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96: Bootstrap starting.
I20260929 18:44:58.281180 937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:58.282184 937 log.cc:824] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:58.284405 937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96: No bootstrap required, opened a new log
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:44:58.288415 937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } }
I20260929 18:44:58.288707 937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:58.288787 937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efb864ee8b54ed4842b1a7adee88c96, State: Initialized, Role: FOLLOWER
I20260929 18:44:58.289247 937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } }
I20260929 18:44:58.289397 937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:58.289485 937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:58.289585 937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:58.290638 937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } }
I20260929 18:44:58.290987 937 leader_election.cc:302] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1efb864ee8b54ed4842b1a7adee88c96; no voters:
I20260929 18:44:58.291352 937 leader_election.cc:288] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:58.291460 942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:58.291672 942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [term 1 LEADER]: Becoming Leader. State: Replica: 1efb864ee8b54ed4842b1a7adee88c96, State: Running, Role: LEADER
I20260929 18:44:58.292033 942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } }
I20260929 18:44:58.292748 937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:58.292795 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:58.278481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50419 (local address 127.11.118.254:39089)
0929 18:44:58.278908 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:58.278931 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:58.278973 (+ 42us) server_negotiation.cc:407] Connection header received
0929 18:44:58.279868 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:58.279892 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:58.280347 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:58.280729 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:58.281512 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:58.282948 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:58.283872 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:58.284376 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:58.284552 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:58.287344 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:58.287350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:58.287365 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:58.287395 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:58.289413 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:58.290018 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:58.290025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:58.290031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:58.290155 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:58.290539 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:58.290542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:58.290544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:58.291006 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:58.291257 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:58.291541 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 18:44:58.291823 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":106,"threads_started":1}
I20260929 18:44:58.293692 943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1efb864ee8b54ed4842b1a7adee88c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } } }
I20260929 18:44:58.293838 943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:58.294072 943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1efb864ee8b54ed4842b1a7adee88c96. Latest consensus state: current_term: 1 leader_uuid: "1efb864ee8b54ed4842b1a7adee88c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1efb864ee8b54ed4842b1a7adee88c96" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 39089 } } }
I20260929 18:44:58.294145 943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:58.294545 951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:58.296165 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:58.297860 951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:58.303933 951 catalog_manager.cc:1347] Generated new cluster ID: be6cd17493b04286a6853c92a151895e
I20260929 18:44:58.304014 951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:58.310150 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:58.298350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56899 (local address 127.11.118.254:39089)
0929 18:44:58.298493 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:58.298500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:58.302614 (+ 4114us) server_negotiation.cc:407] Connection header received
0929 18:44:58.302768 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:58.302772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:58.302818 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:58.302914 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:58.303662 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:58.304276 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:58.305084 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:58.305338 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:58.305443 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:58.306925 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:58.306929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:58.306932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:44:58.306961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:58.308417 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:58.308892 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:58.308895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:58.308897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:58.308977 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:44:58.309282 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:44:58.309286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:44:58.309288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:44:58.309470 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:58.309621 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:58.309885 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 18:44:58.309981 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:44:58.324227 951 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:58.325214 951 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:58.333791 951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1efb864ee8b54ed4842b1a7adee88c96: Generated new TSK 0
I20260929 18:44:58.334605 951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:58.338411 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 869
W20260929 18:44:58.343724 854 connection.cc:535] client connection to 127.11.118.254:39089 recv error: Network error: failed to read from TLS socket (remote: 127.11.118.254:39089): Connection reset by peer (error 104)
2026-09-29T18:44:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[968](info): set up 2 sockets
Sep 29 18:44:58 dist-test-slave-cgv7 krb5kdc[968](info): commencing operation
krb5kdc: starting...
W20260929 18:44:59.382652 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:44:59 dist-test-slave-cgv7 krb5kdc[968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:44:59.505568 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40561
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:40561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:44:59.620534 981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:44:59.620884 981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:44:59.620963 981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:44:59.625172 981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:44:59.625286 981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:44:59.625324 981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:44:59.625361 981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:44:59.625413 981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:44:59.631007 981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:40561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:40561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:44:59.632174 981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:44:59.633246 981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:44:59.642176 989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:59.642375 981 server_base.cc:1034] running on GCE node
W20260929 18:44:59.642217 987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:44:59.642565 986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:44:59.643136 981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:44:59.644584 981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:44:59.645833 981 hybrid_clock.cc:648] HybridClock initialized: now 1790707499645811 us; error 41 us; skew 500 ppm
Sep 29 18:44:59 dist-test-slave-cgv7 krb5kdc[968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707499, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:44:59.649012 981 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:44:59.650003 981 webserver.cc:460] Webserver started at http://127.11.118.254:35767/ using document root <none> and password file <none>
I20260929 18:44:59.650624 981 fs_manager.cc:359] Metadata directory not provided
I20260929 18:44:59.650713 981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:44:59.650941 981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:44:59.653393 981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "f5efd0aa0f3346b6bd9ad778395977f2"
format_stamp: "Formatted at 2026-09-29 18:44:59 on dist-test-slave-cgv7"
server_key: "6fd63594a5b90ee7bce24dfe120d3024"
server_key_iv: "5b43892519e9df306dcd502de474830d"
server_key_version: "encryptionkey@0"
I20260929 18:44:59.653997 981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "f5efd0aa0f3346b6bd9ad778395977f2"
format_stamp: "Formatted at 2026-09-29 18:44:59 on dist-test-slave-cgv7"
server_key: "6fd63594a5b90ee7bce24dfe120d3024"
server_key_iv: "5b43892519e9df306dcd502de474830d"
server_key_version: "encryptionkey@0"
I20260929 18:44:59.657806 981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:44:59.660441 996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:44:59.661722 981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 18:44:59.661870 981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "f5efd0aa0f3346b6bd9ad778395977f2"
format_stamp: "Formatted at 2026-09-29 18:44:59 on dist-test-slave-cgv7"
server_key: "6fd63594a5b90ee7bce24dfe120d3024"
server_key_iv: "5b43892519e9df306dcd502de474830d"
server_key_version: "encryptionkey@0"
I20260929 18:44:59.662019 981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:44:59.677476 981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:44:59.680815 981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:44:59.681030 981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:44:59.688423 981 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:40561
I20260929 18:44:59.688450 1048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:40561 every 8 connection(s)
I20260929 18:44:59.689409 981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:44:59.692440 1049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:44:59.692520 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 981
I20260929 18:44:59.692662 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:44:59.692986 11739 external_mini_cluster.cc:1428] Setting key 45fc1fbe8f9324cd96c867d438271a0e
I20260929 18:44:59.699247 1049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2: Bootstrap starting.
I20260929 18:44:59.701545 1049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2: Neither blocks nor log segments found. Creating new log.
I20260929 18:44:59.702239 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:59.694499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41291 (local address 127.11.118.254:40561)
0929 18:44:59.694943 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:59.694957 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:59.694993 (+ 36us) server_negotiation.cc:407] Connection header received
0929 18:44:59.695794 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:59.695813 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:59.696161 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:59.696516 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:59.696915 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:59.697895 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:59.698871 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:59.699422 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:59.699591 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:59.700545 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:59.700550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:59.700563 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:59.700947 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:59.701001 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:59.701024 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:59.701158 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:59.701338 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 18:44:59.701487 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":143,"threads_started":1}
I20260929 18:44:59.702368 1049 log.cc:824] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2: Log is configured to *not* fsync() on all Append() calls
I20260929 18:44:59.705091 1049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2: No bootstrap required, opened a new log
I20260929 18:44:59.705323 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:44:59.709249 1049 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } }
I20260929 18:44:59.709539 1049 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:44:59.709586 1049 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efd0aa0f3346b6bd9ad778395977f2, State: Initialized, Role: FOLLOWER
I20260929 18:44:59.710047 1049 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } }
I20260929 18:44:59.710175 1049 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:44:59.710256 1049 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:44:59.710376 1049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:44:59.711612 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:44:59.707372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38107 (local address 127.11.118.254:40561)
0929 18:44:59.707480 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 18:44:59.707483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:44:59.707746 (+ 263us) server_negotiation.cc:407] Connection header received
0929 18:44:59.707877 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:44:59.707879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:44:59.707916 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:44:59.707988 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:44:59.708269 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:59.708773 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:59.709858 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:44:59.710130 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:44:59.710272 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:44:59.711159 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:44:59.711162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:44:59.711164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:44:59.711175 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:44:59.711199 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:44:59.711209 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:44:59.711271 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 18:44:59.711437 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 18:44:59.711465 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 18:44:59.711607 1049 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } }
I20260929 18:44:59.712010 1049 leader_election.cc:302] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5efd0aa0f3346b6bd9ad778395977f2; no voters:
I20260929 18:44:59.712354 1049 leader_election.cc:288] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:44:59.712432 1059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:44:59.712620 1059 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [term 1 LEADER]: Becoming Leader. State: Replica: f5efd0aa0f3346b6bd9ad778395977f2, State: Running, Role: LEADER
I20260929 18:44:59.712914 1059 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } }
I20260929 18:44:59.713675 1049 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:44:59.714093 1059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } } }
I20260929 18:44:59.714205 1059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:59.714442 1059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5efd0aa0f3346b6bd9ad778395977f2. Latest consensus state: current_term: 1 leader_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5efd0aa0f3346b6bd9ad778395977f2" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 40561 } } }
I20260929 18:44:59.714550 1059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2 [sys.catalog]: This master's current role is: LEADER
I20260929 18:44:59.714773 1063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:44:59.717612 1063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:44:59.723341 1063 catalog_manager.cc:1347] Generated new cluster ID: 28244281c12a4e3e9777e35fafbf801c
I20260929 18:44:59.723415 1063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:44:59.732789 1063 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:44:59.733701 1063 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:44:59.745309 1063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5efd0aa0f3346b6bd9ad778395977f2: Generated new TSK 0
I20260929 18:44:59.746308 1063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:44:59.755427 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 981
W20260929 18:44:59.760808 966 connection.cc:535] client connection to 127.11.118.254:40561 recv error: Network error: failed to read from TLS socket (remote: 127.11.118.254:40561): Connection reset by peer (error 104)
2026-09-29T18:44:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:44:59 dist-test-slave-cgv7 krb5kdc[1080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:44:59 dist-test-slave-cgv7 krb5kdc[1080](info): set up 2 sockets
Sep 29 18:44:59 dist-test-slave-cgv7 krb5kdc[1080](info): commencing operation
krb5kdc: starting...
W20260929 18:45:00.794620 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:00 dist-test-slave-cgv7 krb5kdc[1080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:00.922848 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:32979
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:32979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:01.034730 1093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:01.035033 1093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:01.035079 1093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:01.039382 1093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:01.039458 1093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:01.039479 1093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:01.039496 1093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:01.039533 1093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:01.044973 1093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:32979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:01.046115 1093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:01.047183 1093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:01.056123 1098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:01.056121 1101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:01.056434 1099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:01.056732 1093 server_base.cc:1034] running on GCE node
I20260929 18:45:01.057353 1093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:01.058621 1093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:01.059818 1093 hybrid_clock.cc:648] HybridClock initialized: now 1790707501059817 us; error 53 us; skew 500 ppm
Sep 29 18:45:01 dist-test-slave-cgv7 krb5kdc[1080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:01.062740 1093 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:01.063714 1093 webserver.cc:460] Webserver started at http://127.11.118.254:35523/ using document root <none> and password file <none>
I20260929 18:45:01.064317 1093 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:01.064409 1093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:01.064639 1093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:01.066831 1093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "5d50c4fdf78f4b46b0afe8c91ff16506"
format_stamp: "Formatted at 2026-09-29 18:45:01 on dist-test-slave-cgv7"
server_key: "d52cccc9ab518c7c7221a311a7c185de"
server_key_iv: "31f0f4b27a4b45724b67618bf4f95ae6"
server_key_version: "encryptionkey@0"
I20260929 18:45:01.067407 1093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "5d50c4fdf78f4b46b0afe8c91ff16506"
format_stamp: "Formatted at 2026-09-29 18:45:01 on dist-test-slave-cgv7"
server_key: "d52cccc9ab518c7c7221a311a7c185de"
server_key_iv: "31f0f4b27a4b45724b67618bf4f95ae6"
server_key_version: "encryptionkey@0"
I20260929 18:45:01.071246 1093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 18:45:01.073938 1108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:01.075328 1093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:45:01.075510 1093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "5d50c4fdf78f4b46b0afe8c91ff16506"
format_stamp: "Formatted at 2026-09-29 18:45:01 on dist-test-slave-cgv7"
server_key: "d52cccc9ab518c7c7221a311a7c185de"
server_key_iv: "31f0f4b27a4b45724b67618bf4f95ae6"
server_key_version: "encryptionkey@0"
I20260929 18:45:01.075656 1093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:01.080458 1093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:01.084107 1093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:01.084304 1093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:01.091692 1093 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:32979
I20260929 18:45:01.091714 1160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:32979 every 8 connection(s)
I20260929 18:45:01.092746 1093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:01.095594 1161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:01.099399 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1093
I20260929 18:45:01.099529 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:01.099834 11739 external_mini_cluster.cc:1428] Setting key ff06e6e3817ba656580b893b8debaff4
I20260929 18:45:01.101815 1161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506: Bootstrap starting.
I20260929 18:45:01.104486 1161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:01.105424 1161 log.cc:824] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:01.105973 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:01.101382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55525 (local address 127.11.118.254:32979)
0929 18:45:01.101879 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:01.101900 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:01.102010 (+ 110us) server_negotiation.cc:407] Connection header received
0929 18:45:01.102808 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:01.102828 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:01.103189 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:01.103500 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:01.104446 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:01.104450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:01.104463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:01.104475 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:01.104533 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:01.104552 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:01.104750 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:01.104916 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 18:45:01.105128 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":162,"threads_started":1}
I20260929 18:45:01.107666 1161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506: No bootstrap required, opened a new log
I20260929 18:45:01.109419 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:01.112071 1161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } }
I20260929 18:45:01.112356 1161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:45:01.112416 1161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d50c4fdf78f4b46b0afe8c91ff16506, State: Initialized, Role: FOLLOWER
I20260929 18:45:01.112911 1161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } }
I20260929 18:45:01.113097 1161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:45:01.113180 1161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:45:01.113289 1161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:01.113623 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:01.111647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37321 (local address 127.11.118.254:32979)
0929 18:45:01.111789 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:01.111794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:01.112009 (+ 215us) server_negotiation.cc:407] Connection header received
0929 18:45:01.112171 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:01.112179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:01.112214 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:01.112296 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:01.113058 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:01.113064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:01.113066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:01.113079 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:01.113112 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:01.113124 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:01.113210 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:01.113468 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 18:45:01.113514 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 18:45:01.114619 1161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } }
I20260929 18:45:01.114984 1161 leader_election.cc:302] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d50c4fdf78f4b46b0afe8c91ff16506; no voters:
I20260929 18:45:01.115334 1161 leader_election.cc:288] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:45:01.115402 1171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:01.115585 1171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [term 1 LEADER]: Becoming Leader. State: Replica: 5d50c4fdf78f4b46b0afe8c91ff16506, State: Running, Role: LEADER
I20260929 18:45:01.115947 1171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } }
I20260929 18:45:01.116744 1161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:01.117847 1173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } } }
I20260929 18:45:01.117789 1172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d50c4fdf78f4b46b0afe8c91ff16506. Latest consensus state: current_term: 1 leader_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d50c4fdf78f4b46b0afe8c91ff16506" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 32979 } } }
I20260929 18:45:01.118068 1172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:01.118368 1173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:01.119309 1180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:01.122254 1180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:45:01.128528 1180 catalog_manager.cc:1347] Generated new cluster ID: a0e606c0940d439c87d5d0cf39db9524
I20260929 18:45:01.128618 1180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:01.146752 1180 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:01.147131 1180 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:01.158174 1180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d50c4fdf78f4b46b0afe8c91ff16506: Generated new TSK 0
I20260929 18:45:01.159155 1180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:45:01.167541 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1093
2026-09-29T18:45:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:45:01 dist-test-slave-cgv7 krb5kdc[1192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:45:01 dist-test-slave-cgv7 krb5kdc[1192](info): set up 2 sockets
Sep 29 18:45:01 dist-test-slave-cgv7 krb5kdc[1192](info): commencing operation
krb5kdc: starting...
W20260929 18:45:02.210561 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:02.337383 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37569
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:37569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:02.453402 1205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:02.453763 1205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:02.453856 1205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:02.457986 1205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:02.458104 1205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:02.458141 1205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:02.458181 1205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:02.458213 1205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:02.463697 1205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:37569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:37569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:02.465029 1205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:02.466135 1205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:02.474989 1210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:02.475032 1211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:02.475035 1213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:02.475150 1205 server_base.cc:1034] running on GCE node
I20260929 18:45:02.475884 1205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:02.477022 1205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:02.478247 1205 hybrid_clock.cc:648] HybridClock initialized: now 1790707502478230 us; error 41 us; skew 500 ppm
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:02.486161 1205 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:02.487339 1205 webserver.cc:460] Webserver started at http://127.11.118.254:33359/ using document root <none> and password file <none>
I20260929 18:45:02.487937 1205 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:02.488009 1205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:02.488190 1205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:02.490430 1205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "4c4a3ab482644b45b7b66e6f1ec2d252"
format_stamp: "Formatted at 2026-09-29 18:45:02 on dist-test-slave-cgv7"
server_key: "223ff0e8619d8bd5c6073b02af2e7d67"
server_key_iv: "1ec2a02c093ef7781dec065a0c25d5ec"
server_key_version: "encryptionkey@0"
I20260929 18:45:02.490995 1205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "4c4a3ab482644b45b7b66e6f1ec2d252"
format_stamp: "Formatted at 2026-09-29 18:45:02 on dist-test-slave-cgv7"
server_key: "223ff0e8619d8bd5c6073b02af2e7d67"
server_key_iv: "1ec2a02c093ef7781dec065a0c25d5ec"
server_key_version: "encryptionkey@0"
I20260929 18:45:02.495018 1205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260929 18:45:02.497663 1220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:02.499156 1205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:45:02.499313 1205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "4c4a3ab482644b45b7b66e6f1ec2d252"
format_stamp: "Formatted at 2026-09-29 18:45:02 on dist-test-slave-cgv7"
server_key: "223ff0e8619d8bd5c6073b02af2e7d67"
server_key_iv: "1ec2a02c093ef7781dec065a0c25d5ec"
server_key_version: "encryptionkey@0"
I20260929 18:45:02.499434 1205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:02.513405 1205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:02.516710 1205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:02.516943 1205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:02.524751 1205 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:37569
I20260929 18:45:02.524761 1272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:37569 every 8 connection(s)
I20260929 18:45:02.525784 1205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:02.528828 1273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:02.534502 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1205
I20260929 18:45:02.534653 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:02.534924 11739 external_mini_cluster.cc:1428] Setting key 0815dac24bb7a1ffec2d11288504574d
I20260929 18:45:02.535065 1273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252: Bootstrap starting.
I20260929 18:45:02.537773 1273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:02.538724 1273 log.cc:824] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:02.541211 1273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252: No bootstrap required, opened a new log
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:45:02.545931 1273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } }
I20260929 18:45:02.546243 1273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:45:02.546365 1273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4a3ab482644b45b7b66e6f1ec2d252, State: Initialized, Role: FOLLOWER
I20260929 18:45:02.546924 1273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } }
I20260929 18:45:02.547106 1273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:45:02.547185 1273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:45:02.547334 1273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:02.548650 1273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } }
I20260929 18:45:02.549153 1273 leader_election.cc:302] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c4a3ab482644b45b7b66e6f1ec2d252; no voters:
I20260929 18:45:02.549588 1273 leader_election.cc:288] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:45:02.549662 1278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:02.549917 1278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [term 1 LEADER]: Becoming Leader. State: Replica: 4c4a3ab482644b45b7b66e6f1ec2d252, State: Running, Role: LEADER
I20260929 18:45:02.550323 1278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } }
I20260929 18:45:02.551151 1273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:02.552047 1279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c4a3ab482644b45b7b66e6f1ec2d252. Latest consensus state: current_term: 1 leader_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } } }
I20260929 18:45:02.552205 1279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:02.552546 1280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4a3ab482644b45b7b66e6f1ec2d252" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 37569 } } }
I20260929 18:45:02.552623 1280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:02.552646 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:02.536382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46747 (local address 127.11.118.254:37569)
0929 18:45:02.536824 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:02.536843 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:02.536892 (+ 49us) server_negotiation.cc:407] Connection header received
0929 18:45:02.537801 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:02.537829 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:02.538327 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:02.538720 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:02.539614 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:02.541082 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:02.542206 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:02.542965 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:02.543137 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:02.546156 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:02.546164 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:02.546182 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:02.546222 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:02.548530 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:02.549171 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:02.549179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:02.549186 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:02.549338 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:02.549665 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:02.549669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:02.549670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:02.550350 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:02.551126 (+ 776us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:02.551284 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 18:45:02.551613 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":116,"threads_started":1}
I20260929 18:45:02.553224 1287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:02.556097 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:02.557044 1287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:45:02.563318 1287 catalog_manager.cc:1347] Generated new cluster ID: cb1774eb1de4483eb7ebaebbb7a55ef2
I20260929 18:45:02.563477 1287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:02.567071 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:02.558460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41005 (local address 127.11.118.254:37569)
0929 18:45:02.558563 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:02.558567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:02.558789 (+ 222us) server_negotiation.cc:407] Connection header received
0929 18:45:02.558904 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:02.558907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:02.558962 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:02.559052 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:02.559872 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:02.560695 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:02.561694 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:02.562077 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:02.562189 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:02.563932 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:02.563936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:02.563939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:02.563968 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:02.565401 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:02.565816 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:02.565819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:02.565821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:02.565909 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:02.566177 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:02.566181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:02.566182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:02.566419 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:02.566545 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:02.566783 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 18:45:02.566906 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 18:45:02.588645 1287 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:02.589618 1287 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:02.601425 1287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c4a3ab482644b45b7b66e6f1ec2d252: Generated new TSK 0
I20260929 18:45:02.602246 1287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:45:02.610853 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1205
2026-09-29T18:45:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1304](info): set up 2 sockets
Sep 29 18:45:02 dist-test-slave-cgv7 krb5kdc[1304](info): commencing operation
krb5kdc: starting...
W20260929 18:45:03.666756 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:03 dist-test-slave-cgv7 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:03.801761 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41643
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:44839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:41643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:03.913664 1317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:03.914003 1317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:03.914084 1317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:03.918170 1317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:03.918280 1317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:03.918339 1317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:03.918378 1317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:03.918412 1317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:03.923797 1317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:44839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:41643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:41643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:03.924924 1317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:03.925990 1317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:03.934634 1325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:03.934623 1323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:03.934628 1322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:03.934679 1317 server_base.cc:1034] running on GCE node
I20260929 18:45:03.935487 1317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:03.936573 1317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:03.937748 1317 hybrid_clock.cc:648] HybridClock initialized: now 1790707503937704 us; error 60 us; skew 500 ppm
Sep 29 18:45:03 dist-test-slave-cgv7 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707503, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:03.940776 1317 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:03.941720 1317 webserver.cc:460] Webserver started at http://127.11.118.254:46741/ using document root <none> and password file <none>
I20260929 18:45:03.942364 1317 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:03.942438 1317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:03.942664 1317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:03.944805 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "54c2ac9f353948218c63cc6f411431df"
format_stamp: "Formatted at 2026-09-29 18:45:03 on dist-test-slave-cgv7"
server_key: "10b7c4b39251b47550bc0bdc1ac50dcf"
server_key_iv: "8c109537609f21b7914c15000c1988b7"
server_key_version: "encryptionkey@0"
I20260929 18:45:03.945477 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "54c2ac9f353948218c63cc6f411431df"
format_stamp: "Formatted at 2026-09-29 18:45:03 on dist-test-slave-cgv7"
server_key: "10b7c4b39251b47550bc0bdc1ac50dcf"
server_key_iv: "8c109537609f21b7914c15000c1988b7"
server_key_version: "encryptionkey@0"
I20260929 18:45:03.949558 1317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 18:45:03.952158 1332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:03.953403 1317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 18:45:03.953552 1317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "54c2ac9f353948218c63cc6f411431df"
format_stamp: "Formatted at 2026-09-29 18:45:03 on dist-test-slave-cgv7"
server_key: "10b7c4b39251b47550bc0bdc1ac50dcf"
server_key_iv: "8c109537609f21b7914c15000c1988b7"
server_key_version: "encryptionkey@0"
I20260929 18:45:03.953660 1317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:03.965560 1317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:03.968678 1317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:03.968876 1317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:03.976191 1317 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:41643
I20260929 18:45:03.976202 1384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:41643 every 8 connection(s)
I20260929 18:45:03.977161 1317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:03.978318 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1317
I20260929 18:45:03.978477 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:03.978765 11739 external_mini_cluster.cc:1428] Setting key 3a9dee99b87b9e5f7a9621f630ef27e5
I20260929 18:45:03.980405 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:03.986963 1385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df: Bootstrap starting.
I20260929 18:45:03.987749 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:03.980217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60547 (local address 127.11.118.254:41643)
0929 18:45:03.980629 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:03.980645 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:03.980680 (+ 35us) server_negotiation.cc:407] Connection header received
0929 18:45:03.981310 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:03.981327 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:03.981669 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:03.981956 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:03.982365 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:03.983346 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:03.984509 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:03.985002 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:03.985112 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:03.985994 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:03.985998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:03.986011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:03.986474 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:03.986529 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:03.986552 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:03.986629 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:03.986785 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 18:45:03.987004 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":266,"thread_start_us":147,"threads_started":1}
I20260929 18:45:03.989212 1385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:03.990090 1385 log.cc:824] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:03.990592 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:03.992180 1385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df: No bootstrap required, opened a new log
I20260929 18:45:03.997273 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:03.992681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36399 (local address 127.11.118.254:41643)
0929 18:45:03.992796 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:03.992801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:03.993067 (+ 266us) server_negotiation.cc:407] Connection header received
0929 18:45:03.993246 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:03.993249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:03.993293 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:03.993387 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:03.993693 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:03.994244 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:03.995505 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:03.995732 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:03.995824 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:03.996668 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:03.996671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:03.996673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:03.996686 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:03.996712 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:03.996722 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:03.996787 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:03.996942 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 18:45:03.997000 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 18:45:03.997088 1385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } }
I20260929 18:45:03.997395 1385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:45:03.997447 1385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c2ac9f353948218c63cc6f411431df, State: Initialized, Role: FOLLOWER
I20260929 18:45:03.997967 1385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } }
I20260929 18:45:03.998134 1385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:45:03.998211 1385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:45:03.998342 1385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:03.999402 1385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } }
I20260929 18:45:03.999730 1385 leader_election.cc:302] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c2ac9f353948218c63cc6f411431df; no voters:
I20260929 18:45:04.000053 1385 leader_election.cc:288] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:45:04.000140 1395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:04.000351 1395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [term 1 LEADER]: Becoming Leader. State: Replica: 54c2ac9f353948218c63cc6f411431df, State: Running, Role: LEADER
I20260929 18:45:04.000768 1395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } }
I20260929 18:45:04.001468 1385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:04.002542 1396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c2ac9f353948218c63cc6f411431df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } } }
I20260929 18:45:04.002678 1396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:04.003043 1397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c2ac9f353948218c63cc6f411431df. Latest consensus state: current_term: 1 leader_uuid: "54c2ac9f353948218c63cc6f411431df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c2ac9f353948218c63cc6f411431df" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 41643 } } }
I20260929 18:45:04.003123 1404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:04.004490 1397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:04.007735 1404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:45:04.013631 1404 catalog_manager.cc:1347] Generated new cluster ID: ac72039c5ef64419a59c1307213cf77f
I20260929 18:45:04.013712 1404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:04.021622 1404 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:04.022665 1404 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:04.029278 1404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c2ac9f353948218c63cc6f411431df: Generated new TSK 0
I20260929 18:45:04.030143 1404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:45:04.044618 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1317
2026-09-29T18:45:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:45:04 dist-test-slave-cgv7 krb5kdc[1416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:45:04 dist-test-slave-cgv7 krb5kdc[1416](info): set up 2 sockets
Sep 29 18:45:04 dist-test-slave-cgv7 krb5kdc[1416](info): commencing operation
krb5kdc: starting...
W20260929 18:45:05.110653 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:05 dist-test-slave-cgv7 krb5kdc[1416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:05.233291 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:35267
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:43893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:05.349648 1429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:05.349957 1429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:05.350028 1429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:05.354144 1429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:05.354230 1429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:05.354250 1429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:05.354287 1429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:05.354303 1429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:05.359829 1429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:43893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:35267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:05.361017 1429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:05.362080 1429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:05.370926 1437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:05.370931 1435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:05.370990 1429 server_base.cc:1034] running on GCE node
W20260929 18:45:05.370934 1434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:05.371718 1429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:05.372898 1429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:05.374070 1429 hybrid_clock.cc:648] HybridClock initialized: now 1790707505374046 us; error 38 us; skew 500 ppm
Sep 29 18:45:05 dist-test-slave-cgv7 krb5kdc[1416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707505, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:05.377099 1429 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:05.378088 1429 webserver.cc:460] Webserver started at http://127.11.118.254:37027/ using document root <none> and password file <none>
I20260929 18:45:05.378733 1429 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:05.378803 1429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:05.379047 1429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:05.381109 1429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "b4238326599f490f8c60c4f2bbdec0ca"
format_stamp: "Formatted at 2026-09-29 18:45:05 on dist-test-slave-cgv7"
server_key: "9a62adaabac7bb0150107fdc938d4573"
server_key_iv: "b96b60a645852c58bfa9a1c7f4667e05"
server_key_version: "encryptionkey@0"
I20260929 18:45:05.381668 1429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "b4238326599f490f8c60c4f2bbdec0ca"
format_stamp: "Formatted at 2026-09-29 18:45:05 on dist-test-slave-cgv7"
server_key: "9a62adaabac7bb0150107fdc938d4573"
server_key_iv: "b96b60a645852c58bfa9a1c7f4667e05"
server_key_version: "encryptionkey@0"
I20260929 18:45:05.385423 1429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 18:45:05.387866 1444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:05.389119 1429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 18:45:05.389256 1429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "b4238326599f490f8c60c4f2bbdec0ca"
format_stamp: "Formatted at 2026-09-29 18:45:05 on dist-test-slave-cgv7"
server_key: "9a62adaabac7bb0150107fdc938d4573"
server_key_iv: "b96b60a645852c58bfa9a1c7f4667e05"
server_key_version: "encryptionkey@0"
I20260929 18:45:05.389384 1429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:05.394068 1429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:05.397059 1429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:05.397271 1429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:05.404384 1429 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:35267
I20260929 18:45:05.404392 1496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:35267 every 8 connection(s)
I20260929 18:45:05.405342 1429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:05.408116 1497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:05.409997 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1429
I20260929 18:45:05.410138 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:05.410459 11739 external_mini_cluster.cc:1428] Setting key b048878090ed912b7a3a55f6b9a76f59
I20260929 18:45:05.414727 1497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: Bootstrap starting.
I20260929 18:45:05.415762 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:05.411858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56845 (local address 127.11.118.254:35267)
0929 18:45:05.412303 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:05.412319 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:05.412373 (+ 54us) server_negotiation.cc:407] Connection header received
0929 18:45:05.413040 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:05.413057 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:05.413337 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:05.413690 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:05.414501 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:05.414505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:05.414517 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:05.414528 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:05.414582 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:05.414601 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:05.414759 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:05.414828 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 18:45:05.415009 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
I20260929 18:45:05.417182 1497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:05.418018 1497 log.cc:824] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:05.418574 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:05.420363 1497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: No bootstrap required, opened a new log
I20260929 18:45:05.422432 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:05.420775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44149 (local address 127.11.118.254:35267)
0929 18:45:05.420900 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:05.420906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:05.421091 (+ 185us) server_negotiation.cc:407] Connection header received
0929 18:45:05.421243 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:05.421245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:05.421282 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:05.421357 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:05.422047 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:05.422050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:05.422052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 18:45:05.422063 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:05.422088 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 18:45:05.422104 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:05.422188 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:05.422260 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 18:45:05.422318 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 18:45:05.424533 1497 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } }
I20260929 18:45:05.424754 1497 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 18:45:05.424835 1497 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4238326599f490f8c60c4f2bbdec0ca, State: Initialized, Role: FOLLOWER
I20260929 18:45:05.425261 1497 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } }
I20260929 18:45:05.425410 1497 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 18:45:05.425493 1497 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 18:45:05.425607 1497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:05.426610 1497 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } }
I20260929 18:45:05.426986 1497 leader_election.cc:302] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4238326599f490f8c60c4f2bbdec0ca; no voters:
I20260929 18:45:05.427349 1497 leader_election.cc:288] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 18:45:05.427426 1507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:05.427608 1507 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [term 1 LEADER]: Becoming Leader. State: Replica: b4238326599f490f8c60c4f2bbdec0ca, State: Running, Role: LEADER
I20260929 18:45:05.427922 1507 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } }
I20260929 18:45:05.428700 1497 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:05.429986 1509 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4238326599f490f8c60c4f2bbdec0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } } }
I20260929 18:45:05.429929 1508 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4238326599f490f8c60c4f2bbdec0ca. Latest consensus state: current_term: 1 leader_uuid: "b4238326599f490f8c60c4f2bbdec0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4238326599f490f8c60c4f2bbdec0ca" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 35267 } } }
I20260929 18:45:05.430143 1508 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:05.430158 1509 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca [sys.catalog]: This master's current role is: LEADER
W20260929 18:45:05.433758 1522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:05.433843 1522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:05.433930 1518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:05.434733 1518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:45:05.440667 1518 catalog_manager.cc:1347] Generated new cluster ID: c25778814c3b4431b17f8154ee264552
I20260929 18:45:05.440749 1518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:05.453477 1518 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:05.453816 1518 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:05.461988 1518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4238326599f490f8c60c4f2bbdec0ca: Generated new TSK 0
I20260929 18:45:05.462874 1518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 18:45:05.474792 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1429
2026-09-29T18:45:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1429 ms)
[----------] 18 tests from AuthTokenIssuingTest (25654 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:45:05 dist-test-slave-cgv7 krb5kdc[1528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:45:05 dist-test-slave-cgv7 krb5kdc[1528](info): set up 2 sockets
Sep 29 18:45:05 dist-test-slave-cgv7 krb5kdc[1528](info): commencing operation
krb5kdc: starting...
W20260929 18:45:06.522650 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:06 dist-test-slave-cgv7 krb5kdc[1528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:06.649638 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34533
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:06.763981 1541 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:06.764308 1541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:06.764333 1541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:06.764374 1541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:06.768456 1541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:06.768522 1541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:06.768543 1541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:06.768560 1541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:06.768575 1541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:06.773910 1541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:34533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:06.774983 1541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:06.775978 1541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 18:45:06.785151 1541 server_base.cc:1034] running on GCE node
W20260929 18:45:06.785091 1546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:06.785126 1549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:06.785091 1547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:06.785990 1541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:06.787184 1541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:06.788372 1541 hybrid_clock.cc:648] HybridClock initialized: now 1790707506788362 us; error 50 us; skew 500 ppm
Sep 29 18:45:06 dist-test-slave-cgv7 krb5kdc[1528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:06.791237 1541 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:06.792161 1541 webserver.cc:460] Webserver started at http://127.11.118.254:40687/ using document root <none> and password file <none>
I20260929 18:45:06.792738 1541 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:06.792830 1541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:06.793067 1541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:06.795269 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "bfd80d62f5944778a83fe602a0c2e02b"
format_stamp: "Formatted at 2026-09-29 18:45:06 on dist-test-slave-cgv7"
server_key: "99bc8cd80a79f0e1d66ad01ccf38090f"
server_key_iv: "6f21f418472840e56058957e24e32124"
server_key_version: "encryptionkey@0"
I20260929 18:45:06.795974 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "bfd80d62f5944778a83fe602a0c2e02b"
format_stamp: "Formatted at 2026-09-29 18:45:06 on dist-test-slave-cgv7"
server_key: "99bc8cd80a79f0e1d66ad01ccf38090f"
server_key_iv: "6f21f418472840e56058957e24e32124"
server_key_version: "encryptionkey@0"
I20260929 18:45:06.800159 1541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:45:06.802726 1556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:06.804092 1541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 18:45:06.804256 1541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "bfd80d62f5944778a83fe602a0c2e02b"
format_stamp: "Formatted at 2026-09-29 18:45:06 on dist-test-slave-cgv7"
server_key: "99bc8cd80a79f0e1d66ad01ccf38090f"
server_key_iv: "6f21f418472840e56058957e24e32124"
server_key_version: "encryptionkey@0"
I20260929 18:45:06.804382 1541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:06.825289 1541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:06.828832 1541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:06.829057 1541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:06.836645 1541 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:34533
I20260929 18:45:06.836864 1608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:34533 every 8 connection(s)
I20260929 18:45:06.837724 1541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:06.840629 1609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:06.844561 1609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:06.846819 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1541
I20260929 18:45:06.847031 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:06.847326 11739 external_mini_cluster.cc:1428] Setting key b396a6f22053dacbfc40fa36e5122325
WARNING: no policy specified for kudu/127.11.118.253@KRBTEST.COM; defaulting to no policy
Sep 29 18:45:06 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.253@KRBTEST.COM" created.
I20260929 18:45:06.861898 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.846933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34533 (local address 127.0.0.1:49324)
0929 18:45:06.847476 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.847530 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:06.848499 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:06.849303 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:06.849312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:06.849879 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:06.850304 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:06.850329 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:06.851771 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:06.851776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:06.853164 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:06.853179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:06.853880 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:06.853885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:06.854025 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:06.854658 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:06.854684 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:06.856517 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:06.858954 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:06.858963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:06.858975 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:06.859301 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:06.859640 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:06.859646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:06.859651 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:06.859808 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:06.860467 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:06.860478 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:06.860634 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 18:45:06.860781 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 18:45:06.861081 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":164,"threads_started":1}
I20260929 18:45:06.861907 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.847888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.11.118.254:34533)
0929 18:45:06.848227 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:06.848241 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:06.848288 (+ 47us) server_negotiation.cc:407] Connection header received
0929 18:45:06.848870 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:06.848876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:06.849078 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:06.849242 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:06.850538 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:06.851605 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:06.853313 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:06.853681 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:06.853814 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:06.856684 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:06.856695 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:06.856711 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:06.856757 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:06.858835 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:06.859439 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:06.859444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:06.859445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:06.859549 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:06.859906 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:06.859908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:06.859909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:06.860312 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:06.860490 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:06.860910 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 18:45:06.861214 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260929 18:45:06.864511 1609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:06.865255 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.864987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58776)
0929 18:45:06.865111 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.865189 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 18:45:06.865483 1558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
I20260929 18:45:06.866398 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.866164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58792)
0929 18:45:06.866294 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.866339 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 18:45:06.866532 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:33387: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
I20260929 18:45:06.877640 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:33387
--webserver_interface=127.11.118.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:45:06.915794 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } attempt: 1
I20260929 18:45:06.916790 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.916483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58796)
0929 18:45:06.916637 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.916711 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 18:45:06.917629 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.917406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58810)
0929 18:45:06.917518 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.917573 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 18:45:06.917799 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:33387: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
I20260929 18:45:06.985960 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } attempt: 2
I20260929 18:45:06.986860 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.986563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58812)
0929 18:45:06.986721 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.986787 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 18:45:06.987746 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:06.987508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58820)
0929 18:45:06.987633 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:06.987690 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 18:45:06.987937 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:33387: Network error: Client connection negotiation failed: client connection to 127.11.118.253:33387: connect: Connection refused (error 111)
W20260929 18:45:06.991267 1616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:06.991596 1616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:06.991650 1616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:06.991721 1616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:06.996011 1616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:06.996071 1616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:06.996093 1616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:06.996132 1616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:06.996163 1616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:07.002086 1616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:33387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.118.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:07.003342 1616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:07.004515 1616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:07.013571 1622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:07.013576 1621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:07.013566 1624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:07.013854 1616 server_base.cc:1034] running on GCE node
I20260929 18:45:07.014694 1616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:07.015795 1616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:07.016991 1616 hybrid_clock.cc:648] HybridClock initialized: now 1790707507016956 us; error 49 us; skew 500 ppm
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:07.019888 1616 init.cc:377] Logged in from keytab as kudu/127.11.118.253@KRBTEST.COM (short username kudu)
I20260929 18:45:07.020740 1616 webserver.cc:460] Webserver started at http://127.11.118.253:39843/ using document root <none> and password file <none>
I20260929 18:45:07.021276 1616 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:07.021335 1616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:07.021507 1616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:07.023595 1616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data/instance:
uuid: "6b65ac7d50b84e2a941d46a954a6c851"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "0b3abb505083ec66d1059bc2dcfb3207"
server_key_iv: "5dfca4b3314303d5e6b180b9bef02280"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.024092 1616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal/instance:
uuid: "6b65ac7d50b84e2a941d46a954a6c851"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "0b3abb505083ec66d1059bc2dcfb3207"
server_key_iv: "5dfca4b3314303d5e6b180b9bef02280"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.027949 1616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 18:45:07.030227 1631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:07.031450 1616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 18:45:07.031572 1616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "6b65ac7d50b84e2a941d46a954a6c851"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "0b3abb505083ec66d1059bc2dcfb3207"
server_key_iv: "5dfca4b3314303d5e6b180b9bef02280"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.031674 1616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:07.055544 1616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:07.058604 1616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:07.058821 1616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:07.065569 1616 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.253:33387
I20260929 18:45:07.065877 1683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.253:33387 every 8 connection(s)
I20260929 18:45:07.066581 1616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
I20260929 18:45:07.069322 1684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:07.073035 1684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.074863 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1616
I20260929 18:45:07.074995 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-1/wal/instance
I20260929 18:45:07.075266 11739 external_mini_cluster.cc:1428] Setting key 2110917a7aa9c64cfb2fb1e8f6d1182d
I20260929 18:45:07.079103 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } attempt: 3
WARNING: no policy specified for kudu/127.11.118.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.252@KRBTEST.COM" created.
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
I20260929 18:45:07.092254 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.079603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58832)
0929 18:45:07.079795 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.079810 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.079898 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.080606 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.080609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.080806 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.080919 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.080926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.082236 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.082240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.083112 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.083123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.083984 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.083988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.084106 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.084618 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.084639 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.088060 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.090303 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.090307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.090309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.090541 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.090957 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.090963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.090968 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.091077 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.091752 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.091755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.091858 (+ 103us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.091970 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.092124 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260929 18:45:07.092648 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.074917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.11.118.254:34533)
0929 18:45:07.075100 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.075104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.076135 (+ 1031us) server_negotiation.cc:407] Connection header received
0929 18:45:07.077452 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.077455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.077511 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.077592 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.079387 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.080279 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.081917 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.082345 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.082465 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.087688 (+ 5223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.087692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.087694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.087743 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.090491 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.091065 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.091072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.091076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.091164 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.091483 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.091489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.091494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.091739 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.091832 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.092397 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.092511 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 18:45:07.093768 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.079719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.11.118.253:33387)
0929 18:45:07.080139 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.080155 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.080174 (+ 19us) server_negotiation.cc:407] Connection header received
0929 18:45:07.080207 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.080211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.080399 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.081069 (+ 670us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.081191 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.082129 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.083235 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.083841 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.083956 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.088257 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.088266 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.088294 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.088324 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.090153 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.090671 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.090680 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.090685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.090847 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.091181 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.091184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.091185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.091633 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.091777 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.092920 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.093076 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"spinlock_wait_cycles":27008,"thread_start_us":205,"threads_started":1}
I20260929 18:45:07.093781 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.075444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34533 (local address 127.0.0.1:49326)
0929 18:45:07.075952 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.076010 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.077156 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.077838 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.077852 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.078418 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.079175 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.079199 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.080417 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.080420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.081780 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.081789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.082510 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.082514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.082730 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.083645 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.083667 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.087506 (+ 3839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.090610 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.090621 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.090635 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.090865 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.091275 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.091278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.091280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.091381 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.091925 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.091931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.092109 (+ 178us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.092315 (+ 206us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.092678 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":128,"threads_started":1}
I20260929 18:45:07.095781 1684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.095858 1609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.096784 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.096568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55078)
0929 18:45:07.096677 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.096725 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 18:45:07.097517 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.097265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55084)
0929 18:45:07.097397 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.097452 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 18:45:07.097687 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
I20260929 18:45:07.104593 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.096305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58838)
0929 18:45:07.096786 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.096802 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.096888 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.097133 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.097136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.097300 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.097433 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.097440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.098373 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.098378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.099161 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.099170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.099693 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.099696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.099856 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.100240 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.100250 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.101449 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.103159 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.103162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.103163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.103446 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.103681 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.103683 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.103685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.103756 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.104160 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.104162 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.104241 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.104322 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.104423 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":36}
I20260929 18:45:07.104806 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.096374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.11.118.253:33387)
0929 18:45:07.096487 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.096491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.096851 (+ 360us) server_negotiation.cc:407] Connection header received
0929 18:45:07.096993 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.096997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.097030 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.097123 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.097538 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.098227 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.099280 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.099588 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.099696 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.101565 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.101568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.101570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.101589 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.103054 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.103531 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.103533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.103535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.103604 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.103875 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.103881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.103885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.104079 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.104155 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.104401 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.104552 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":41}
I20260929 18:45:07.105660 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:45043
--webserver_interface=127.11.118.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:45:07.105679 1684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.106535 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.106175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55100)
0929 18:45:07.106384 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.106476 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 18:45:07.107080 1632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
I20260929 18:45:07.108009 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.107740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55114)
0929 18:45:07.107917 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.107958 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260929 18:45:07.108174 1684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
I20260929 18:45:07.128877 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 1
I20260929 18:45:07.129810 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.129462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55128)
0929 18:45:07.129630 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.129704 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 18:45:07.130676 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.130459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55136)
0929 18:45:07.130563 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.130620 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 18:45:07.130904 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
I20260929 18:45:07.157440 1684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 1
I20260929 18:45:07.158457 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.158092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55152)
0929 18:45:07.158256 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.158374 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 18:45:07.159196 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.158974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55156)
0929 18:45:07.159082 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.159137 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 18:45:07.159343 1684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
I20260929 18:45:07.206101 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 2
I20260929 18:45:07.207079 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.206747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55166)
0929 18:45:07.206925 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.206995 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 18:45:07.207825 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.207611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55168)
0929 18:45:07.207716 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.207770 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 18:45:07.207993 1609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
W20260929 18:45:07.220458 1690 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:07.220844 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:07.220906 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:07.220984 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:07.225217 1690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:07.225317 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:07.225353 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:07.225391 1690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:07.225436 1690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:07.227501 1684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 2
I20260929 18:45:07.228291 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.228006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55184)
0929 18:45:07.228146 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.228214 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 18:45:07.229029 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.228787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55186)
0929 18:45:07.228936 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.228979 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 18:45:07.229146 1684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:45043: Network error: Client connection negotiation failed: client connection to 127.11.118.252:45043: connect: Connection refused (error 111)
I20260929 18:45:07.231287 1690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:34533,127.11.118.253:33387,127.11.118.252:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:45043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.118.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:07.232434 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:07.233484 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:07.242651 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:07.242718 1690 server_base.cc:1034] running on GCE node
W20260929 18:45:07.242643 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:07.242643 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:07.243666 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:07.244868 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:07.246091 1690 hybrid_clock.cc:648] HybridClock initialized: now 1790707507246075 us; error 43 us; skew 500 ppm
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:07.248982 1690 init.cc:377] Logged in from keytab as kudu/127.11.118.252@KRBTEST.COM (short username kudu)
I20260929 18:45:07.249962 1690 webserver.cc:460] Webserver started at http://127.11.118.252:45451/ using document root <none> and password file <none>
I20260929 18:45:07.250587 1690 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:07.250654 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:07.250821 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:07.253064 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data/instance:
uuid: "ac5a92499465430cabe8b42062e5c328"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "6d814ff11a59eab5accd5adfac1a4bc0"
server_key_iv: "312f94d06d6502a9e32c4ea1f6936ec1"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.253624 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal/instance:
uuid: "ac5a92499465430cabe8b42062e5c328"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "6d814ff11a59eab5accd5adfac1a4bc0"
server_key_iv: "312f94d06d6502a9e32c4ea1f6936ec1"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.257445 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:45:07.259972 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:07.261615 1690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:45:07.261742 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "ac5a92499465430cabe8b42062e5c328"
format_stamp: "Formatted at 2026-09-29 18:45:07 on dist-test-slave-cgv7"
server_key: "6d814ff11a59eab5accd5adfac1a4bc0"
server_key_iv: "312f94d06d6502a9e32c4ea1f6936ec1"
server_key_version: "encryptionkey@0"
I20260929 18:45:07.261864 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:07.279470 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:07.282670 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:07.282889 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:07.290087 1690 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.252:45043
I20260929 18:45:07.290433 1758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.252:45043 every 8 connection(s)
I20260929 18:45:07.291198 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
I20260929 18:45:07.292761 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1690
I20260929 18:45:07.293009 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790707223158922-11739-0/minicluster-data/master-2/wal/instance
I20260929 18:45:07.293363 11739 external_mini_cluster.cc:1428] Setting key 47ab65db3073c09f86e770f5863061ea
I20260929 18:45:07.294709 1759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:07.300220 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:45:07.307154 1609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 3
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:45:07.311007 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.294815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.11.118.252:45043)
0929 18:45:07.296351 (+ 1536us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.296370 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.296415 (+ 45us) server_negotiation.cc:407] Connection header received
0929 18:45:07.297285 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.297310 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.297751 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.298162 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.298992 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.300547 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.301543 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.302121 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.302334 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.305473 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.305478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.305488 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.305531 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.307487 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.308027 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.308035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.308041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.308192 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.308527 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.308532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.308537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.309116 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.309344 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.309667 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.310038 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260929 18:45:07.313271 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.302516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.11.118.254:34533)
0929 18:45:07.302642 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.302647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.303309 (+ 662us) server_negotiation.cc:407] Connection header received
0929 18:45:07.303458 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.303460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.303505 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.303575 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.304488 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.305169 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.306314 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.306682 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.306798 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.309383 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.309388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.309401 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.309428 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.311056 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.311593 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.311597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.311599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.311693 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.312267 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.312271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.312274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.312491 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.312552 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.312990 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.313121 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:45:07.314059 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.302354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:34533 (local address 127.0.0.1:49328)
0929 18:45:07.303155 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.303216 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.303364 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.303668 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.303673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.304198 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.304355 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.304361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.305299 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.305302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.306167 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.306176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.306804 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.306807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.306891 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.307398 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.307417 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.309232 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.311182 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.311191 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.311197 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.311484 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.311804 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.311807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.311809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.312152 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.312620 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.312625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.312746 (+ 121us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.312905 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.313102 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":159,"threads_started":1}
I20260929 18:45:07.315510 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.317773 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.307616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55202)
0929 18:45:07.307811 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.307828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.307904 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.308358 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.308361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.308559 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.308659 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.308665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.310064 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.310069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.311061 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.311067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.311666 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.311670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.311765 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.312184 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.312199 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.313700 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.316094 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.316097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.316099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.316352 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.316682 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.316688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.316690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.316789 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.317247 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.317250 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.317350 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.317463 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.317607 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260929 18:45:07.317807 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.307650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.11.118.252:45043)
0929 18:45:07.308128 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.308137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.308148 (+ 11us) server_negotiation.cc:407] Connection header received
0929 18:45:07.308178 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.308180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.308223 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.308768 (+ 545us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.308900 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.309947 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.311182 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.311529 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.311655 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.313838 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.313843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.313850 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.313871 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.315967 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.316479 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.316485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.316487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.316574 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.316902 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.316905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.316907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.317147 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.317248 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.317515 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.317625 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":108,"threads_started":1}
I20260929 18:45:07.320281 1684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } attempt: 3
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
I20260929 18:45:07.322893 1609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: Bootstrap starting.
I20260929 18:45:07.325412 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.314093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.11.118.254:34533)
0929 18:45:07.314193 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.314197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.314334 (+ 137us) server_negotiation.cc:407] Connection header received
0929 18:45:07.314704 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.314708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.314750 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.314835 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.315628 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.316975 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.318108 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.318479 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.318581 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.321423 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.321428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.321431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.321458 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.323412 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.323941 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.323944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.323946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.324032 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.324351 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.324357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.324359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.324584 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.324789 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.325126 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.325250 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:45:07.326982 1609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:07.328016 1609 log.cc:824] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:07.329201 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.316060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:33387 (local address 127.0.0.1:58846)
0929 18:45:07.316155 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.316170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.316243 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.316531 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.316532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.316705 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.316827 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.316837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.318245 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.318250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.319025 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.319036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.319689 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.319692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.319787 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.321454 (+ 1667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.321469 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.323090 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.326898 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.326903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.326907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.327184 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.328178 (+ 994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.328182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.328184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.328276 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.328743 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.328747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.328864 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.328959 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.329077 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 18:45:07.329468 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.316158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.11.118.253:33387)
0929 18:45:07.316292 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.316297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.316313 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:45:07.316356 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.316359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.316428 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.316512 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.316968 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.317707 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.319155 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.319545 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.319678 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.323249 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.323253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.323255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.323282 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.325210 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.327624 (+ 2414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.327628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.327630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.327716 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.328399 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.328403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.328405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.328617 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.329069 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.329184 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.329296 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:45:07.330780 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:07.331599 1609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: No bootstrap required, opened a new log
I20260929 18:45:07.332902 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.320723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55218)
0929 18:45:07.320825 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.320841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.320930 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.321251 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.321254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.321435 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.321536 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.321543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.322747 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.322750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.323641 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.323653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.324283 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.324287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.324383 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.325353 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.325370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.326755 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.330522 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.330526 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.330529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.330803 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.331234 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.331237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.331239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.331326 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.332330 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.332335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.332445 (+ 110us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.332543 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.332700 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
I20260929 18:45:07.333571 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.320948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.11.118.252:45043)
0929 18:45:07.321058 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.321064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.321082 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:45:07.321108 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.321110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.321145 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.321236 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.321675 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.322606 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.323788 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.324163 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.324271 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.327326 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.327329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.327332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.327355 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.330385 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.331037 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.331040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.331041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.331132 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.331454 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.331460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.331462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.332204 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.333205 (+ 1001us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.333290 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.333421 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 18:45:07.336576 1609 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.336840 1609 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd80d62f5944778a83fe602a0c2e02b, State: Initialized, Role: FOLLOWER
Sep 29 18:45:07 dist-test-slave-cgv7 krb5kdc[1528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707507, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:45:07.337203 1684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: Bootstrap starting.
I20260929 18:45:07.337188 1609 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.338670 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.326810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.11.118.253:33387)
0929 18:45:07.327221 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.327231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.327246 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:45:07.327284 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.327287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.327324 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.327426 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.328411 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.329290 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.330544 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.330933 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.331029 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.334232 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.334238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.334243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.334304 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.336325 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.336859 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.336862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.336864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.336947 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.337257 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.337260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.337262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.337503 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.337660 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.338322 (+ 662us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.338460 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":130,"threads_started":1}
I20260929 18:45:07.338861 1609 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:07.339898 1684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:07.340659 1684 log.cc:824] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:07.341646 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.331479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:45043 (local address 127.0.0.1:55220)
0929 18:45:07.331715 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:07.331729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:07.331804 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:07.332049 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:07.332052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:07.332224 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:07.332323 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.332330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.333548 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.333554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.334442 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:07.334453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.335422 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.335426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:07.335501 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.335901 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:07.335916 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:07.337520 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:07.339920 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.339924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.339928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.340235 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.340526 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:07.340529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:07.340530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:07.340604 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:07.341159 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:07.341161 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:07.341257 (+ 96us) client_negotiation.cc:769] Sending connection context
0929 18:45:07.341350 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 18:45:07.341489 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260929 18:45:07.342514 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.331699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.11.118.252:45043)
0929 18:45:07.331820 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.331824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.331842 (+ 18us) server_negotiation.cc:407] Connection header received
0929 18:45:07.331879 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.331885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.331949 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.332033 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.332442 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.333433 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.334963 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.335309 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.335709 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.337787 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.337791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.337793 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.337817 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.339795 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.340361 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.340363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.340365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.340437 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.340708 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.340712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.340716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.341054 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.342136 (+ 1082us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.342211 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.342366 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:45:07.343731 1684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: No bootstrap required, opened a new log
I20260929 18:45:07.344242 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.344378 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: This master's current role is: FOLLOWER
W20260929 18:45:07.344419 1779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:07.344486 1779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:07.345849 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:07.347241 1583 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bfd80d62f5944778a83fe602a0c2e02b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49342
I20260929 18:45:07.347503 1583 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 18:45:07.347633 1583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:07.348032 1684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.348268 1684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b65ac7d50b84e2a941d46a954a6c851, State: Initialized, Role: FOLLOWER
I20260929 18:45:07.348606 1684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.345017 1541 master_runner.cc:186] Error getting master registration for 127.11.118.253:33387: OK, instance_id { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" instance_seqno: 1790707507063682 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:45:07.348986 1541 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:45:07.349004 1583 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.349248 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.349576 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.349999 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: Bootstrap starting.
I20260929 18:45:07.350266 1684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:07.350855 1583 leader_election.cc:288] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [CANDIDATE]: Term 1 election: Requested vote from peers 6b65ac7d50b84e2a941d46a954a6c851 (127.11.118.253:33387), ac5a92499465430cabe8b42062e5c328 (127.11.118.252:45043)
I20260929 18:45:07.351274 1658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfd80d62f5944778a83fe602a0c2e02b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b65ac7d50b84e2a941d46a954a6c851"
I20260929 18:45:07.351547 1658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:07.353430 1759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:07.353612 1658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd80d62f5944778a83fe602a0c2e02b in term 1.
I20260929 18:45:07.354094 1558 leader_election.cc:302] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b65ac7d50b84e2a941d46a954a6c851, bfd80d62f5944778a83fe602a0c2e02b; no voters:
I20260929 18:45:07.354501 1759 log.cc:824] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:07.354768 1767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:07.355008 1767 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [term 1 LEADER]: Becoming Leader. State: Replica: bfd80d62f5944778a83fe602a0c2e02b, State: Running, Role: LEADER
I20260929 18:45:07.355361 1767 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.357355 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd80d62f5944778a83fe602a0c2e02b. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.357496 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:07.358096 1802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:07.359148 1802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260929 18:45:07.359503 1798 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:07.359586 1798 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:07.365103 1658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [term 1 FOLLOWER]: Refusing update from remote peer bfd80d62f5944778a83fe602a0c2e02b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:45:07.365798 1797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:45:07.362627 1616 master_runner.cc:186] Error getting master registration for 127.11.118.252:45043: OK, instance_id { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" instance_seqno: 1790707507287971 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:45:07.366215 1616 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:45:07.367053 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: No bootstrap required, opened a new log
I20260929 18:45:07.370047 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd80d62f5944778a83fe602a0c2e02b. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.370214 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.371291 1759 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.371578 1759 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5a92499465430cabe8b42062e5c328, State: Initialized, Role: FOLLOWER
I20260929 18:45:07.371930 1759 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } }
I20260929 18:45:07.372443 1807 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.372589 1807 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.373322 1759 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:07.373581 1733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:07.373638 1731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfd80d62f5944778a83fe602a0c2e02b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ac5a92499465430cabe8b42062e5c328"
I20260929 18:45:07.375029 1733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [term 1 FOLLOWER]: Refusing update from remote peer bfd80d62f5944778a83fe602a0c2e02b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:45:07.376020 1797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:45:07.376382 1731 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd80d62f5944778a83fe602a0c2e02b in term 1.
I20260929 18:45:07.381052 1807 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd80d62f5944778a83fe602a0c2e02b. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.381217 1807 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.386191 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.362727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.11.118.254:34533)
0929 18:45:07.362833 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.362837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.375542 (+ 12705us) server_negotiation.cc:407] Connection header received
0929 18:45:07.375750 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.375754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.375820 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.375878 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.377362 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.378172 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.379408 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.379757 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.379849 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.382106 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.382110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.382113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.382144 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.384048 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.384552 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.384556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.384558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.384647 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.385030 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.385034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.385035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.385273 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.385473 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.385865 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.386019 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 18:45:07.383450 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.355131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.11.118.252:45043)
0929 18:45:07.355343 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.355348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.362988 (+ 7640us) server_negotiation.cc:407] Connection header received
0929 18:45:07.363143 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.363146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.363195 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.363281 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.365127 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.365910 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.367349 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.367709 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.367814 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.378619 (+ 10805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.378623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.378626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.378655 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.380664 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.381830 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.381834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.381836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.381931 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.382256 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.382261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.382263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.382547 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.382684 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.383083 (+ 399us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.383232 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260929 18:45:07.393769 1821 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:07.394019 1821 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:07.394037 1797 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.394184 1797 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:07.395678 1805 mvcc.cc:205] Tried to move back new op lower bound from 7334737950159470592 to 7334737950131920896. Current Snapshot: MvccSnapshot[applied={T|T < 7334737950159470592}]
I20260929 18:45:07.395852 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.395977 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.396486 1803 mvcc.cc:205] Tried to move back new op lower bound from 7334737950159470592 to 7334737950131920896. Current Snapshot: MvccSnapshot[applied={T|T < 7334737950159470592}]
I20260929 18:45:07.396423 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.396574 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:07.397644 1819 mvcc.cc:205] Tried to move back new op lower bound from 7334737950159470592 to 7334737950131920896. Current Snapshot: MvccSnapshot[applied={T|T < 7334737950159470592}]
I20260929 18:45:07.397756 1807 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfd80d62f5944778a83fe602a0c2e02b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd80d62f5944778a83fe602a0c2e02b" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 34533 } } peers { permanent_uuid: "6b65ac7d50b84e2a941d46a954a6c851" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 33387 } } peers { permanent_uuid: "ac5a92499465430cabe8b42062e5c328" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 45043 } } }
I20260929 18:45:07.397866 1807 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:07.399888 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:07.365397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.11.118.253:33387)
0929 18:45:07.365511 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.365515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.380764 (+ 15249us) server_negotiation.cc:407] Connection header received
0929 18:45:07.381012 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.381017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.381083 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.381144 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.382071 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.382941 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.385717 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:07.386115 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:07.386196 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:07.390481 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:07.390486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:07.390489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:07.390519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:07.392384 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.398337 (+ 5953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.398341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.398343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.398468 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:07.398834 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:07.398838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:07.398840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:07.399077 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:07.399181 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:07.399555 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 18:45:07.399709 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 18:45:07.408260 1802 catalog_manager.cc:1347] Generated new cluster ID: 224eb6e7e714402bb627a5e5ec3c718c
I20260929 18:45:07.408349 1802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:07.436610 1802 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:07.439874 1802 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:07.456032 1802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd80d62f5944778a83fe602a0c2e02b: Generated new TSK 0
I20260929 18:45:07.457069 1802 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:45:07.488042 1612 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.485119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.11.118.254:34533)
0929 18:45:07.485250 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.485255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.486911 (+ 1656us) server_negotiation.cc:407] Connection header received
0929 18:45:07.487160 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.487163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.487216 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.487273 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.487899 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49364: BlockingRecv error: recv got EOF from 127.0.0.1:49364 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260929 18:45:07.489562 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.485913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.11.118.253:33387)
0929 18:45:07.486025 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.486029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.488689 (+ 2660us) server_negotiation.cc:407] Connection header received
0929 18:45:07.488864 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.488867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.488917 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.488969 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.489455 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58876: BlockingRecv error: recv got EOF from 127.0.0.1:58876 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 18:45:07.491163 1762 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.486637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.11.118.252:45043)
0929 18:45:07.486749 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.486754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.490175 (+ 3421us) server_negotiation.cc:407] Connection header received
0929 18:45:07.490446 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.490450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.490499 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.490553 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.491046 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55244: BlockingRecv error: recv got EOF from 127.0.0.1:55244 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 18:45:07.495404 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.493753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.11.118.253:33387)
0929 18:45:07.493870 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.493875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.494378 (+ 503us) server_negotiation.cc:407] Connection header received
0929 18:45:07.494652 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.494655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.494739 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.494803 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.495296 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58888: BlockingRecv error: recv got EOF from 127.0.0.1:58888 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 18:45:07.500774 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.499298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.11.118.253:33387)
0929 18:45:07.499403 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.499407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.499869 (+ 462us) server_negotiation.cc:407] Connection header received
0929 18:45:07.500052 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.500055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.500105 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.500158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.500664 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58890: BlockingRecv error: recv got EOF from 127.0.0.1:58890 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 18:45:07.506754 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.505141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.11.118.253:33387)
0929 18:45:07.505248 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.505252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.505741 (+ 489us) server_negotiation.cc:407] Connection header received
0929 18:45:07.505945 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.505948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.506001 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.506052 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.506646 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58900: BlockingRecv error: recv got EOF from 127.0.0.1:58900 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 18:45:07.515300 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.513443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.11.118.253:33387)
0929 18:45:07.513606 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.513611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.514197 (+ 586us) server_negotiation.cc:407] Connection header received
0929 18:45:07.514465 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.514469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.514528 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.514584 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.515177 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58908: BlockingRecv error: recv got EOF from 127.0.0.1:58908 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 18:45:07.528185 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.526345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.11.118.253:33387)
0929 18:45:07.526473 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.526478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.527122 (+ 644us) server_negotiation.cc:407] Connection header received
0929 18:45:07.527342 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.527345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.527405 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.527463 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.528056 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58910: BlockingRecv error: recv got EOF from 127.0.0.1:58910 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260929 18:45:07.549165 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.547270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.11.118.253:33387)
0929 18:45:07.547410 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.547415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.548075 (+ 660us) server_negotiation.cc:407] Connection header received
0929 18:45:07.548305 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.548308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.548366 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.548422 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.549039 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58912: BlockingRecv error: recv got EOF from 127.0.0.1:58912 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 18:45:07.585273 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.583960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.11.118.253:33387)
0929 18:45:07.584090 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.584094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.584348 (+ 254us) server_negotiation.cc:407] Connection header received
0929 18:45:07.584519 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.584523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.584585 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.584676 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.585143 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58922: BlockingRecv error: recv got EOF from 127.0.0.1:58922 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 18:45:07.653232 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.651882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.11.118.253:33387)
0929 18:45:07.652050 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.652055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.652259 (+ 204us) server_negotiation.cc:407] Connection header received
0929 18:45:07.652354 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.652356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.652424 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.652498 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.653104 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58936: BlockingRecv error: recv got EOF from 127.0.0.1:58936 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 18:45:07.784736 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:07.783415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.11.118.253:33387)
0929 18:45:07.783564 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:07.783569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:07.783800 (+ 231us) server_negotiation.cc:407] Connection header received
0929 18:45:07.783898 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:07.783901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:07.783960 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:07.784034 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:07.784594 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58950: BlockingRecv error: recv got EOF from 127.0.0.1:58950 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 18:45:08.044013 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:08.043019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.11.118.253:33387)
0929 18:45:08.043156 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.043159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.043174 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:45:08.043312 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.043314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.043360 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.043437 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:08.043914 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58960: BlockingRecv error: recv got EOF from 127.0.0.1:58960 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 18:45:08.361017 1798 catalog_manager.cc:1259] Loaded cluster ID: 224eb6e7e714402bb627a5e5ec3c718c
I20260929 18:45:08.361111 1798 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: loading cluster ID for follower catalog manager: success
I20260929 18:45:08.362749 1798 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: acquiring CA information for follower catalog manager: success
I20260929 18:45:08.363911 1798 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b65ac7d50b84e2a941d46a954a6c851: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:45:08.395465 1821 catalog_manager.cc:1259] Loaded cluster ID: 224eb6e7e714402bb627a5e5ec3c718c
I20260929 18:45:08.395551 1821 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: loading cluster ID for follower catalog manager: success
I20260929 18:45:08.397163 1821 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: acquiring CA information for follower catalog manager: success
I20260929 18:45:08.398343 1821 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ac5a92499465430cabe8b42062e5c328: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:45:08.562171 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:08.558175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.11.118.253:33387)
0929 18:45:08.558487 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.558490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.558505 (+ 15us) server_negotiation.cc:407] Connection header received
0929 18:45:08.558559 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.558561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.558682 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.558781 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:08.559114 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.559667 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.560626 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.560853 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.560938 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:08.561265 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:08.561714 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:08.561804 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:08.561959 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 18:45:08.562011 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":119,"threads_started":1}
I20260929 18:45:08.569542 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:08.565512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.11.118.252:45043)
0929 18:45:08.565756 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.565761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.565896 (+ 135us) server_negotiation.cc:407] Connection header received
0929 18:45:08.566095 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.566099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.566226 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.566411 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:08.566632 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.567153 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.568014 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.568261 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.568377 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:08.568695 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:08.569135 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:08.569290 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:08.569324 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 18:45:08.569382 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260929 18:45:08.577214 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:08.573224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.11.118.253:33387)
0929 18:45:08.573362 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.573366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.573510 (+ 144us) server_negotiation.cc:407] Connection header received
0929 18:45:08.573615 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.573618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.573675 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.573744 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:08.574059 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.574812 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.575987 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.576320 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.576425 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:08.576696 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:08.576778 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:08.576848 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:08.576993 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 18:45:08.577063 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:45:08.577996 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:08.572955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.11.118.254:34533)
0929 18:45:08.573203 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.573207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.573510 (+ 303us) server_negotiation.cc:407] Connection header received
0929 18:45:08.573626 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.573628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.573789 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.573944 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:08.574358 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.574972 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.576217 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.576514 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.576615 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:08.577023 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:08.577435 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:08.577515 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:08.577795 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 18:45:08.577856 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260929 18:45:08.578404 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:08.573048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.11.118.252:45043)
0929 18:45:08.573165 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:08.573169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:08.573739 (+ 570us) server_negotiation.cc:407] Connection header received
0929 18:45:08.573944 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:08.573948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:08.573999 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:08.574067 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:08.575234 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.576017 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.577101 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:08.577409 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:08.577498 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:08.577875 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:08.577980 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:08.578028 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:08.578191 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 18:45:08.578262 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 18:45:08.585178 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1541
I20260929 18:45:08.591588 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1616
I20260929 18:45:08.597740 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1690
2026-09-29T18:45:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3125 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 18:45:08 dist-test-slave-cgv7 krb5kdc[1908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 18:45:08 dist-test-slave-cgv7 krb5kdc[1908](info): set up 2 sockets
Sep 29 18:45:08 dist-test-slave-cgv7 krb5kdc[1908](info): commencing operation
krb5kdc: starting...
W20260929 18:45:09.638702 11739 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/test-user.keytab.
Sep 29 18:45:09 dist-test-slave-cgv7 krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T18:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T18:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.118.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
Entry for principal kudu/127.11.118.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab.
I20260929 18:45:09.770156 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:44811
--webserver_interface=127.11.118.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 18:45:09.885694 1921 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:09.886080 1921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:09.886162 1921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:09.886247 1921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:09.890372 1921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:09.890483 1921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:09.890520 1921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:09.890563 1921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:09.890599 1921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:09.896018 1921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.254.keytab
--principal=kudu/127.11.118.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.254:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.118.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:09.897190 1921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:09.898252 1921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:09.907033 1927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:09.907032 1926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:09.907081 1929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:09.907465 1921 server_base.cc:1034] running on GCE node
I20260929 18:45:09.908224 1921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:09.909317 1921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:09.910478 1921 hybrid_clock.cc:648] HybridClock initialized: now 1790707509910461 us; error 33 us; skew 500 ppm
Sep 29 18:45:09 dist-test-slave-cgv7 krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:09.913251 1921 init.cc:377] Logged in from keytab as kudu/127.11.118.254@KRBTEST.COM (short username kudu)
I20260929 18:45:09.914168 1921 webserver.cc:460] Webserver started at http://127.11.118.254:34113/ using document root <none> and password file <none>
I20260929 18:45:09.914750 1921 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:09.914815 1921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:09.914996 1921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:09.917045 1921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data/instance:
uuid: "03b9136a690d4eef9b2d4831826b72b5"
format_stamp: "Formatted at 2026-09-29 18:45:09 on dist-test-slave-cgv7"
server_key: "e4e4e7f00d60777038bc66aa3a4a9bd9"
server_key_iv: "19c239763f147a751f61d42194acafa9"
server_key_version: "encryptionkey@0"
I20260929 18:45:09.917588 1921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance:
uuid: "03b9136a690d4eef9b2d4831826b72b5"
format_stamp: "Formatted at 2026-09-29 18:45:09 on dist-test-slave-cgv7"
server_key: "e4e4e7f00d60777038bc66aa3a4a9bd9"
server_key_iv: "19c239763f147a751f61d42194acafa9"
server_key_version: "encryptionkey@0"
I20260929 18:45:09.921773 1921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 18:45:09.924598 1936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:09.926019 1921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:45:09.926190 1921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal
uuid: "03b9136a690d4eef9b2d4831826b72b5"
format_stamp: "Formatted at 2026-09-29 18:45:09 on dist-test-slave-cgv7"
server_key: "e4e4e7f00d60777038bc66aa3a4a9bd9"
server_key_iv: "19c239763f147a751f61d42194acafa9"
server_key_version: "encryptionkey@0"
I20260929 18:45:09.926368 1921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:09.946820 1921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:09.949834 1921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:09.950037 1921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:09.957856 1921 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.254:44811
I20260929 18:45:09.958077 1988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.254:44811 every 8 connection(s)
I20260929 18:45:09.959179 1921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/data/info.pb
I20260929 18:45:09.961906 1989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:09.965665 1989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:09.967094 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1921
I20260929 18:45:09.967213 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-0/wal/instance
I20260929 18:45:09.967489 11739 external_mini_cluster.cc:1428] Setting key cececdda274a5d5a12964c801060b1f3
WARNING: no policy specified for kudu/127.11.118.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.253@KRBTEST.COM" created.
Sep 29 18:45:09 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:45:09.983251 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:09.968323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:44811 (local address 127.0.0.1:59062)
0929 18:45:09.969101 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:09.969138 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:09.969985 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:09.970696 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:09.970701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:09.971139 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:09.971451 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:09.971465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:09.972598 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:09.972601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:09.973301 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:09.973309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:09.973943 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:09.973947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:09.974040 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:09.974645 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:09.974661 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:09.978331 (+ 3670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:09.980279 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:09.980289 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:09.980299 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:09.980533 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:09.980852 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:09.980854 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:09.980856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:09.980972 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:09.981564 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:09.981570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:09.981709 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 18:45:09.981875 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 18:45:09.982262 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":105,"spinlock_wait_cycles":60544,"thread_start_us":165,"threads_started":1}
I20260929 18:45:09.983251 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:09.968323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.11.118.254:44811)
0929 18:45:09.968849 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:09.968872 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:09.969247 (+ 375us) server_negotiation.cc:407] Connection header received
0929 18:45:09.970311 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:09.970316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:09.970521 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:09.970700 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:09.971601 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:09.972458 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:09.973418 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:09.973792 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:09.973956 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:09.978487 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:09.978493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:09.978502 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:09.978523 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:09.980152 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:09.980642 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:09.980647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:09.980649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:09.980753 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:09.981068 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:09.981071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:09.981073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:09.981446 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:09.981582 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:09.981947 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 18:45:09.982187 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":165,"threads_started":1}
I20260929 18:45:09.985601 1989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:09.986548 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:09.986181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51670)
0929 18:45:09.986346 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:09.986462 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260929 18:45:09.986752 1938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
I20260929 18:45:09.987553 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:09.987326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51676)
0929 18:45:09.987436 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:09.987486 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 18:45:09.987687 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:34717: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
Entry for principal kudu/127.11.118.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab.
I20260929 18:45:09.994189 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:34717
--webserver_interface=127.11.118.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:45:10.036984 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } attempt: 1
I20260929 18:45:10.037940 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.037637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51684)
0929 18:45:10.037789 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.037854 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 18:45:10.038805 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.038553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51698)
0929 18:45:10.038678 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.038737 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 18:45:10.038936 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:34717: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
I20260929 18:45:10.107098 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } attempt: 2
I20260929 18:45:10.108002 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.107636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51712)
0929 18:45:10.107807 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.107873 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 18:45:10.108836 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.108586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51722)
0929 18:45:10.108722 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.108775 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 18:45:10.109055 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:34717: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
W20260929 18:45:10.110096 1996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:10.110436 1996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:10.110491 1996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:10.110541 1996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:10.114758 1996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:10.114814 1996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:10.114835 1996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:10.114852 1996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:10.114868 1996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:10.120311 1996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.253.keytab
--principal=kudu/127.11.118.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.253:34717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.118.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:10.121451 1996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:10.122503 1996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:10.131284 2002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:10.131263 2001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:10.131268 2004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:10.131603 1996 server_base.cc:1034] running on GCE node
I20260929 18:45:10.132251 1996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:10.133450 1996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:10.134642 1996 hybrid_clock.cc:648] HybridClock initialized: now 1790707510134607 us; error 52 us; skew 500 ppm
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:10.137403 1996 init.cc:377] Logged in from keytab as kudu/127.11.118.253@KRBTEST.COM (short username kudu)
I20260929 18:45:10.138422 1996 webserver.cc:460] Webserver started at http://127.11.118.253:33741/ using document root <none> and password file <none>
I20260929 18:45:10.138949 1996 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:10.139019 1996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:10.139184 1996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:10.141214 1996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data/instance:
uuid: "4c503ee067d548669c96f1e13024a841"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "9cd09551bfaa44474555732b71a20a75"
server_key_iv: "dd65894d100c7be2dcf8459862440c65"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.141718 1996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal/instance:
uuid: "4c503ee067d548669c96f1e13024a841"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "9cd09551bfaa44474555732b71a20a75"
server_key_iv: "dd65894d100c7be2dcf8459862440c65"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.145543 1996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 18:45:10.148149 2011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:10.149585 1996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 18:45:10.149698 1996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal
uuid: "4c503ee067d548669c96f1e13024a841"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "9cd09551bfaa44474555732b71a20a75"
server_key_iv: "dd65894d100c7be2dcf8459862440c65"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.149868 1996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:10.191901 1996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:10.194903 1996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:10.195128 1996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:10.200227 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } attempt: 3
I20260929 18:45:10.201071 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.200769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51732)
0929 18:45:10.200905 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.200976 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 18:45:10.201795 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.201567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51742)
0929 18:45:10.201676 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.201730 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 18:45:10.201943 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.253:34717: Network error: Client connection negotiation failed: client connection to 127.11.118.253:34717: connect: Connection refused (error 111)
I20260929 18:45:10.202623 1996 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.253:34717
I20260929 18:45:10.202893 2063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.253:34717 every 8 connection(s)
I20260929 18:45:10.203622 1996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/data/info.pb
I20260929 18:45:10.206436 2064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:10.210394 2064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:10.211567 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 1996
I20260929 18:45:10.211694 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-1/wal/instance
I20260929 18:45:10.212021 11739 external_mini_cluster.cc:1428] Setting key b6fabf7b95806e6d6f7f59015b88205f
WARNING: no policy specified for kudu/127.11.118.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.118.252@KRBTEST.COM" created.
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:45:10.231098 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.212637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.11.118.254:44811)
0929 18:45:10.212817 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.212825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.213700 (+ 875us) server_negotiation.cc:407] Connection header received
0929 18:45:10.214808 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.214811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.214864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.214953 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.216674 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.217211 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.219136 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.219397 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.219499 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.225492 (+ 5993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.225496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.225498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.225559 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.228310 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.229064 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.229068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.229070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.229179 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.229609 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.229615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.229620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.229960 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.230092 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.230742 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.230894 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 18:45:10.231984 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.213026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:44811 (local address 127.0.0.1:59064)
0929 18:45:10.213518 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.213573 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.214543 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.215168 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.215180 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.215721 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.216475 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.216500 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.217357 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.217364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.218994 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.219006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.219584 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.219591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.219826 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.220498 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.220527 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.225218 (+ 4691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.228478 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.228484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.228501 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.228892 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.229298 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.229300 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.229302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.229469 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.230091 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.230096 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.230459 (+ 363us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.230657 (+ 198us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.231008 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":144,"threads_started":1}
I20260929 18:45:10.233479 2064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
Entry for principal kudu/127.11.118.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab.
I20260929 18:45:10.242112 11739 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqumnYE/build/debug/bin/kudu
/tmp/dist-test-taskqumnYE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:35085
--webserver_interface=127.11.118.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.118.212:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 18:45:10.243042 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.233958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51756)
0929 18:45:10.234094 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.234106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.234170 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.234926 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.234929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.235106 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.235199 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.235205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.236185 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.236191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.236820 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.236829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.237475 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.237479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.237564 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.238082 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.238094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.239333 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.241252 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.241258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.241264 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.241544 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.241867 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.241870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.241871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.241952 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.242510 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.242513 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.242608 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.242716 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.242858 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 18:45:10.243093 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.234069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.11.118.253:34717)
0929 18:45:10.234539 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.234578 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.234605 (+ 27us) server_negotiation.cc:407] Connection header received
0929 18:45:10.234640 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.234644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.234801 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.234945 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.235303 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.236076 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.236947 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.237361 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.237482 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.239486 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.239495 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.239508 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.239531 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.241145 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.241676 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.241682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.241684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.241769 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.242047 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.242052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.242054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.242404 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.242539 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.242812 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.242958 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":153,"threads_started":1}
I20260929 18:45:10.245555 2064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:10.246529 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.246115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48480)
0929 18:45:10.246359 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.246454 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260929 18:45:10.246742 2015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
I20260929 18:45:10.247807 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.247571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48482)
0929 18:45:10.247697 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.247750 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 18:45:10.248088 2064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:35085: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
I20260929 18:45:10.297395 2064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } attempt: 1
I20260929 18:45:10.298481 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.298125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48494)
0929 18:45:10.298330 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.298400 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 18:45:10.299365 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.299106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48504)
0929 18:45:10.299233 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.299302 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 18:45:10.299561 2064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:35085: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
I20260929 18:45:10.345149 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } attempt: 4
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
I20260929 18:45:10.355438 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.345866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.11.118.253:34717)
0929 18:45:10.346022 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.346027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.346046 (+ 19us) server_negotiation.cc:407] Connection header received
0929 18:45:10.346181 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.346184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.346235 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.346365 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.346811 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.348079 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.349001 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.349379 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.349498 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.351988 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.351993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.351995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.352023 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.353706 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.354217 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.354221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.354222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.354343 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.354653 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.354656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.354657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.354836 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.354911 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.355204 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.355303 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 18:45:10.355477 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.345778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51772)
0929 18:45:10.345972 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.345990 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.346082 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.346421 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.346424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.346591 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.346690 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.346694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.348242 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.348246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.348855 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.348865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.349521 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.349524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.349615 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.350212 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.350224 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.351830 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.353830 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.353834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.353838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.354096 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.354445 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.354448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.354451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.354556 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.354924 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.354926 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.355048 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.355157 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.355309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 18:45:10.356364 1989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:10.357143 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.356866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48506)
0929 18:45:10.357026 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.357079 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 18:45:10.357896 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.357700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48512)
0929 18:45:10.357794 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.357843 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 18:45:10.358107 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:35085: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
W20260929 18:45:10.360018 2070 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 18:45:10.360369 2070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 18:45:10.360427 2070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 18:45:10.360507 2070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 18:45:10.364796 2070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 18:45:10.364872 2070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 18:45:10.364892 2070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 18:45:10.364910 2070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 18:45:10.364926 2070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 18:45:10.367779 2064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } attempt: 2
I20260929 18:45:10.368685 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.368373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48526)
0929 18:45:10.368542 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.368606 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 18:45:10.369573 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.369338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48540)
0929 18:45:10.369463 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.369515 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 18:45:10.369694 2064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:35085: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
I20260929 18:45:10.371079 2070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.118.212:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.118.254:44811,127.11.118.253:34717,127.11.118.252:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/krb5kdc/kudu_127.11.118.252.keytab
--principal=kudu/127.11.118.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.118.252:35085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.118.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 17:54:46 UTC on ccycloud.quasar-azztjd.root.comops.site
build id 4680912
I20260929 18:45:10.372219 2070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 18:45:10.373240 2070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 18:45:10.381979 2079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:10.381983 2076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 18:45:10.382016 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 18:45:10.382206 2070 server_base.cc:1034] running on GCE node
I20260929 18:45:10.382798 2070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 18:45:10.383939 2070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 18:45:10.385141 2070 hybrid_clock.cc:648] HybridClock initialized: now 1790707510385125 us; error 29 us; skew 500 ppm
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 18:45:10.387957 2070 init.cc:377] Logged in from keytab as kudu/127.11.118.252@KRBTEST.COM (short username kudu)
I20260929 18:45:10.388873 2070 webserver.cc:460] Webserver started at http://127.11.118.252:39791/ using document root <none> and password file <none>
I20260929 18:45:10.389452 2070 fs_manager.cc:359] Metadata directory not provided
I20260929 18:45:10.389546 2070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 18:45:10.389804 2070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 18:45:10.391975 2070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data/instance:
uuid: "d04b2238e2474d919b3cb4a7d6ce58c1"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "52a90c618350285ed070182b23983777"
server_key_iv: "7718fa5b18de663cab43e62d426719d4"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.392549 2070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal/instance:
uuid: "d04b2238e2474d919b3cb4a7d6ce58c1"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "52a90c618350285ed070182b23983777"
server_key_iv: "7718fa5b18de663cab43e62d426719d4"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.396359 2070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 18:45:10.398835 2086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 18:45:10.400095 2070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 18:45:10.400236 2070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data,/tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal
uuid: "d04b2238e2474d919b3cb4a7d6ce58c1"
format_stamp: "Formatted at 2026-09-29 18:45:10 on dist-test-slave-cgv7"
server_key: "52a90c618350285ed070182b23983777"
server_key_iv: "7718fa5b18de663cab43e62d426719d4"
server_key_version: "encryptionkey@0"
I20260929 18:45:10.400352 2070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 18:45:10.417297 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } attempt: 1
I20260929 18:45:10.418236 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.417897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48544)
0929 18:45:10.418088 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.418154 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 18:45:10.419090 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.418868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48558)
0929 18:45:10.418982 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.419037 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 18:45:10.419217 1989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.118.252:35085: Network error: Client connection negotiation failed: client connection to 127.11.118.252:35085: connect: Connection refused (error 111)
I20260929 18:45:10.424834 2070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 18:45:10.427868 2070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 18:45:10.428076 2070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 18:45:10.435125 2070 rpc_server.cc:307] RPC server started. Bound to: 127.11.118.252:35085
I20260929 18:45:10.435402 2138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.118.252:35085 every 8 connection(s)
I20260929 18:45:10.436129 2070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/data/info.pb
I20260929 18:45:10.438897 2139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 18:45:10.439787 11739 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqumnYE/build/debug/bin/kudu as pid 2070
I20260929 18:45:10.439919 11739 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqumnYE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790707223158922-11739-0/minicluster-data/master-2/wal/instance
I20260929 18:45:10.440251 11739 external_mini_cluster.cc:1428] Setting key 7883264ba97a0274fa5a320109b21d5d
I20260929 18:45:10.443646 2139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
I20260929 18:45:10.456068 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.445558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.11.118.254:44811)
0929 18:45:10.445700 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.445704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.445999 (+ 295us) server_negotiation.cc:407] Connection header received
0929 18:45:10.446125 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.446128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.446179 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.446264 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.447057 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.447540 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.448892 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.449313 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.449422 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.452796 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.452799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.452802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.452833 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.454364 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.454802 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.454806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.454808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.454888 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.455187 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.455190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.455192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.455414 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.455521 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.455815 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.455932 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 18:45:10.456251 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.441558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.11.118.252:35085)
0929 18:45:10.441989 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.442008 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.442051 (+ 43us) server_negotiation.cc:407] Connection header received
0929 18:45:10.442912 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.442935 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.443355 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.443718 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.444442 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.445766 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.446853 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.447332 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.447510 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.450559 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.450570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.450602 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.450634 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.452643 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.453467 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.453472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.453477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.453623 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.453947 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.453952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.453964 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.454455 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.454665 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.454967 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.455258 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260929 18:45:10.456255 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.445489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.254:44811 (local address 127.0.0.1:59076)
0929 18:45:10.445863 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.445910 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.446032 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.446375 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.446379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.446807 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.446928 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.446936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.447658 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.447662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.448752 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.448763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.449592 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.449598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.449723 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.451046 (+ 1323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.451066 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.452646 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.454466 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.454476 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.454482 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.454702 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.454988 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.454991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.454993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.455096 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.455527 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.455532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.455625 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.455738 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.455864 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":139,"threads_started":1}
I20260929 18:45:10.457443 2139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:10.460857 2064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } attempt: 3
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.254@KRBTEST.COM
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.253@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
I20260929 18:45:10.468828 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.458789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59084 (local address 127.11.118.254:44811)
0929 18:45:10.458915 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.458918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.458930 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:45:10.458968 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.458970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.459011 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.459081 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.459634 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.460377 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.461313 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.461991 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.462102 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.464789 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.464794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.464798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.464824 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.466476 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.466975 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.466979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.466981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.467072 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.467376 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.467380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.467382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.467610 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.467854 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.468542 (+ 688us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.468667 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:45:10.468860 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.458081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.11.118.253:34717)
0929 18:45:10.458198 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.458205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.458222 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:45:10.458286 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.458290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.458337 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.458413 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.458921 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.459518 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.460635 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.460968 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.461052 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.465302 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.465309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.465314 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.465347 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.467171 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.467617 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.467620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.467622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.467703 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.467983 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.467986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.467988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.468196 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.468303 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.468625 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.468729 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 18:45:10.469303 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.457950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.253:34717 (local address 127.0.0.1:51776)
0929 18:45:10.458076 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.458091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.458166 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.458522 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.458524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.458651 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.458781 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.458787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.459663 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.459666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.460489 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.460497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.461265 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.461269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.461369 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.461878 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.461891 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.464934 (+ 3043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.467250 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.467254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.467256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.467513 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.467800 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.467803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.467805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.467888 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.468352 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.468355 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.468454 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.468570 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.468702 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 18:45:10.470167 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.461271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48584)
0929 18:45:10.461545 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.461560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.461635 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.461850 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.461853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.462027 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.462140 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.462146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.463160 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.463165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.463672 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.463679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.464131 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.464135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.464215 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.464649 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.464659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.466140 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.468271 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.468278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.468281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.468516 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.468799 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.468802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.468804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.468885 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.469663 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.469666 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.469781 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.469865 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.470005 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260929 18:45:10.470491 2139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 18:45:10.470795 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.461504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.11.118.252:35085)
0929 18:45:10.461610 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.461614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.461626 (+ 12us) server_negotiation.cc:407] Connection header received
0929 18:45:10.461710 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.461714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.461749 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.461835 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.462237 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.463071 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.463782 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.464023 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.464127 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.466541 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.466548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.466553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.466578 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.468176 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.468620 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.468623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.468624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.468698 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.469013 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.469017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.469020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.469547 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.470461 (+ 914us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.470528 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.470647 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 18:45:10.474105 2064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: Bootstrap starting.
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707510, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.252@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.118.253@KRBTEST.COM
I20260929 18:45:10.477329 2064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:10.478022 2064 log.cc:824] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:10.480077 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.470983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48594)
0929 18:45:10.471309 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.471322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.471403 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.471712 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.471715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.471859 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.471958 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.471966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.472864 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.472867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.473635 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.473644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.474175 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.474181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.474346 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.474784 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.474802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.476324 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.478372 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.478379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.478383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.478654 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.478961 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.478965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.478966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.479051 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.479505 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.479508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.479588 (+ 80us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.479717 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.479877 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260929 18:45:10.480309 2064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: No bootstrap required, opened a new log
I20260929 18:45:10.480114 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.470990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.11.118.252:35085)
0929 18:45:10.471074 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.471079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.471391 (+ 312us) server_negotiation.cc:407] Connection header received
0929 18:45:10.471481 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.471487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.471533 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.471616 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.472074 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.472745 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.473751 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.474049 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.474154 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.476453 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.476464 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.476466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.476487 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.478208 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.478766 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.478770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.478772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.478852 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.479144 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.479148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.479149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.479404 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.479517 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.479774 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.479956 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":32}
I20260929 18:45:10.482677 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.470203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.11.118.253:34717)
0929 18:45:10.470361 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.470367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.470931 (+ 564us) server_negotiation.cc:407] Connection header received
0929 18:45:10.471092 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.471095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.471134 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.471227 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.471990 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.472504 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.473502 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.473819 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.473873 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.477119 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.477123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.477125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.477152 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.479187 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.479667 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.479673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.479678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.479759 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.480062 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.480065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.480067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.480286 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.480448 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.482420 (+ 1972us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.482538 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 18:45:10.483035 2139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: Bootstrap starting.
I20260929 18:45:10.483870 2064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.484112 2064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c503ee067d548669c96f1e13024a841, State: Initialized, Role: FOLLOWER
I20260929 18:45:10.484458 2064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.485529 2145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.485690 2145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.485726 2139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:10.485755 2064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:10.486197 11739 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 18:45:10.486372 1989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } attempt: 2
I20260929 18:45:10.486630 2139 log.cc:824] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:10.487604 1963 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "03b9136a690d4eef9b2d4831826b72b5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59084
I20260929 18:45:10.489200 2139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: No bootstrap required, opened a new log
I20260929 18:45:10.493564 2139 raft_consensus.cc:348] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.493815 2139 raft_consensus.cc:729] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04b2238e2474d919b3cb4a7d6ce58c1, State: Initialized, Role: FOLLOWER
I20260929 18:45:10.494174 2139 consensus_queue.cc:260] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
Sep 29 18:45:10 dist-test-slave-cgv7 krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790707509, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.118.254@KRBTEST.COM for kudu/127.11.118.252@KRBTEST.COM
W20260929 18:45:10.495291 2157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:10.495491 2157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:10.495859 2139 sys_catalog.cc:564] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:10.498724 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.486754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.118.252:35085 (local address 127.0.0.1:48600)
0929 18:45:10.487002 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 18:45:10.487017 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 18:45:10.487094 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 18:45:10.489152 (+ 2058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 18:45:10.489155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 18:45:10.489359 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0929 18:45:10.489454 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.489461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.490446 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.490449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.491304 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 18:45:10.491316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.491965 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.491969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 18:45:10.492069 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.492604 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 18:45:10.492622 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 18:45:10.494938 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 18:45:10.496968 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.496972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.496974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.497239 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.497572 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 18:45:10.497575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 18:45:10.497577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 18:45:10.497665 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 18:45:10.498131 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 18:45:10.498134 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 18:45:10.498236 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 18:45:10.498388 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 18:45:10.498547 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260929 18:45:10.499320 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.487251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.11.118.252:35085)
0929 18:45:10.488919 (+ 1668us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.488923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.488942 (+ 19us) server_negotiation.cc:407] Connection header received
0929 18:45:10.488991 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.488996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.489047 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.489136 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.489583 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.490328 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.491443 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.491754 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.491849 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.495107 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.495111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.495114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.495139 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.496833 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.497356 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.497361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.497362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.497448 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.497770 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.497773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.497775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.497995 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.498087 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.499051 (+ 964us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.499173 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1564}
I20260929 18:45:10.490883 1996 master_runner.cc:186] Error getting master registration for 127.11.118.254:44811: OK, instance_id { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" instance_seqno: 1790707509955661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:45:10.500408 1996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:45:10.500625 2159 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.500746 2159 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.502794 1989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: Bootstrap starting.
I20260929 18:45:10.499979 2070 master_runner.cc:186] Error getting master registration for 127.11.118.254:44811: OK, instance_id { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" instance_seqno: 1790707509955661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 18:45:10.504252 2070 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:45:10.506062 1989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: Neither blocks nor log segments found. Creating new log.
I20260929 18:45:10.506911 1989 log.cc:824] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: Log is configured to *not* fsync() on all Append() calls
I20260929 18:45:10.509543 1989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: No bootstrap required, opened a new log
W20260929 18:45:10.510360 2170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:10.510447 2170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:10.512874 1989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.513149 1989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b9136a690d4eef9b2d4831826b72b5, State: Initialized, Role: FOLLOWER
I20260929 18:45:10.513520 1989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.514923 1989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 18:45:10.515030 1963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 18:45:10.515169 1963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:10.516517 1963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.517242 2173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.517725 2173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.518213 2038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03b9136a690d4eef9b2d4831826b72b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4c503ee067d548669c96f1e13024a841"
I20260929 18:45:10.518514 2038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:10.519833 2038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b9136a690d4eef9b2d4831826b72b5 in term 1.
I20260929 18:45:10.520397 2113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03b9136a690d4eef9b2d4831826b72b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1"
I20260929 18:45:10.520694 2113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 18:45:10.522635 2113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b9136a690d4eef9b2d4831826b72b5 in term 1.
I20260929 18:45:10.524331 1938 leader_election.cc:302] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b9136a690d4eef9b2d4831826b72b5, 4c503ee067d548669c96f1e13024a841; no voters:
W20260929 18:45:10.526553 2184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 18:45:10.526727 2184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 18:45:10.527396 1963 leader_election.cc:288] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [CANDIDATE]: Term 1 election: Requested vote from peers 4c503ee067d548669c96f1e13024a841 (127.11.118.253:34717), d04b2238e2474d919b3cb4a7d6ce58c1 (127.11.118.252:35085)
I20260929 18:45:10.527840 2173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 18:45:10.528069 2173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [term 1 LEADER]: Becoming Leader. State: Replica: 03b9136a690d4eef9b2d4831826b72b5, State: Running, Role: LEADER
I20260929 18:45:10.529071 1921 master_runner.cc:263] No leader master found from master 03b9136a690d4eef9b2d4831826b72b5
I20260929 18:45:10.529140 1921 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 18:45:10.529220 2173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } }
I20260929 18:45:10.531499 2173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03b9136a690d4eef9b2d4831826b72b5. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.531616 2173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:10.535123 2194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 18:45:10.536146 2194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 18:45:10.549788 2113 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [term 1 FOLLOWER]: Refusing update from remote peer 03b9136a690d4eef9b2d4831826b72b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:45:10.550675 2186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:45:10.553237 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.531110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.11.118.252:35085)
0929 18:45:10.531216 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.531220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.536838 (+ 5618us) server_negotiation.cc:407] Connection header received
0929 18:45:10.536930 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.536933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.536991 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.537083 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.538944 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.539751 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.541071 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.541485 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.541600 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.543833 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.543837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.543840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.543871 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.550931 (+ 7060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.551562 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.551566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.551568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.551660 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.551934 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.551938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.551940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.552154 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.552275 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.552957 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.553078 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 18:45:10.556599 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.534710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.11.118.253:34717)
0929 18:45:10.534837 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.534841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.542693 (+ 7852us) server_negotiation.cc:407] Connection header received
0929 18:45:10.542890 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.542894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.542951 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.543009 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.545080 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.545879 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.547135 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.547508 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.547592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.549463 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.549467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.549470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.549503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.551477 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.555102 (+ 3625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.555107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.555108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.555207 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.555520 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.555523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.555525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.555743 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.555868 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.556259 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.556390 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 18:45:10.557430 2159 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03b9136a690d4eef9b2d4831826b72b5. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.557554 2159 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.559440 2038 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [term 1 FOLLOWER]: Refusing update from remote peer 03b9136a690d4eef9b2d4831826b72b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 18:45:10.559985 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:10.530654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.11.118.254:44811)
0929 18:45:10.530790 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.530795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.536487 (+ 5692us) server_negotiation.cc:407] Connection header received
0929 18:45:10.536647 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.536650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.536707 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.536791 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.542705 (+ 5914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.543540 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.549421 (+ 5881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:10.549802 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:10.549881 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:10.552724 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 18:45:10.552728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 18:45:10.552730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 18:45:10.552757 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 18:45:10.554700 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.558506 (+ 3806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.558510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.558512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.558614 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 18:45:10.558964 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 18:45:10.558967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 18:45:10.558969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 18:45:10.559191 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 18:45:10.559304 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:10.559664 (+ 360us) server_negotiation.cc:299] Negotiation successful
0929 18:45:10.559796 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:45:10.560098 2186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 18:45:10.564458 2145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03b9136a690d4eef9b2d4831826b72b5. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.564595 2145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.568688 2186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.568827 2186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:10.569934 2145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.570057 2145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.570461 2196 mvcc.cc:205] Tried to move back new op lower bound from 7334737963201888256 to 7334737963131617280. Current Snapshot: MvccSnapshot[applied={T|T < 7334737963201888256}]
I20260929 18:45:10.573868 2173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.574012 2173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5 [sys.catalog]: This master's current role is: LEADER
I20260929 18:45:10.574946 2159 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03b9136a690d4eef9b2d4831826b72b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b9136a690d4eef9b2d4831826b72b5" member_type: VOTER last_known_addr { host: "127.11.118.254" port: 44811 } } peers { permanent_uuid: "4c503ee067d548669c96f1e13024a841" member_type: VOTER last_known_addr { host: "127.11.118.253" port: 34717 } } peers { permanent_uuid: "d04b2238e2474d919b3cb4a7d6ce58c1" member_type: VOTER last_known_addr { host: "127.11.118.252" port: 35085 } } }
I20260929 18:45:10.575109 2159 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 18:45:10.576719 2199 mvcc.cc:205] Tried to move back new op lower bound from 7334737963201888256 to 7334737963131617280. Current Snapshot: MvccSnapshot[applied={T|T < 7334737963201888256}]
I20260929 18:45:10.576719 2197 mvcc.cc:205] Tried to move back new op lower bound from 7334737963201888256 to 7334737963131617280. Current Snapshot: MvccSnapshot[applied={T|T < 7334737963201888256}]
I20260929 18:45:10.576964 2194 catalog_manager.cc:1347] Generated new cluster ID: 87ce17bce65e478a9bf005cac95490c4
I20260929 18:45:10.577028 2194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 18:45:10.618878 2194 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 18:45:10.620237 2194 catalog_manager.cc:1504] Loading token signing keys...
I20260929 18:45:10.636547 2194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03b9136a690d4eef9b2d4831826b72b5: Generated new TSK 0
I20260929 18:45:10.637454 2194 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 18:45:10.679404 1991 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.676040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59114 (local address 127.11.118.254:44811)
0929 18:45:10.676171 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.676176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.678010 (+ 1834us) server_negotiation.cc:407] Connection header received
0929 18:45:10.678473 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.678477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.678534 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.678589 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.679259 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59114: BlockingRecv error: recv got EOF from 127.0.0.1:59114 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 18:45:10.680533 2142 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.676787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48610 (local address 127.11.118.252:35085)
0929 18:45:10.676901 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.676905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.678381 (+ 1476us) server_negotiation.cc:407] Connection header received
0929 18:45:10.679762 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.679765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.679816 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.679865 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.680410 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48610: BlockingRecv error: recv got EOF from 127.0.0.1:48610 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 18:45:10.682050 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.677479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.11.118.253:34717)
0929 18:45:10.677592 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.677596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.681026 (+ 3430us) server_negotiation.cc:407] Connection header received
0929 18:45:10.681343 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.681347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.681399 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.681453 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.681919 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51804: BlockingRecv error: recv got EOF from 127.0.0.1:51804 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 18:45:10.685878 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.684383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.11.118.253:34717)
0929 18:45:10.684484 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.684489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.684919 (+ 430us) server_negotiation.cc:407] Connection header received
0929 18:45:10.685151 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.685154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.685230 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.685283 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.685783 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51808: BlockingRecv error: recv got EOF from 127.0.0.1:51808 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 18:45:10.691254 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.689634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.11.118.253:34717)
0929 18:45:10.689743 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.689747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.690258 (+ 511us) server_negotiation.cc:407] Connection header received
0929 18:45:10.690511 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.690514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.690566 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.690621 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.691148 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51810: BlockingRecv error: recv got EOF from 127.0.0.1:51810 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 18:45:10.697559 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.695692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51818 (local address 127.11.118.253:34717)
0929 18:45:10.695809 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.695815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.696464 (+ 649us) server_negotiation.cc:407] Connection header received
0929 18:45:10.696698 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.696702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.696754 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.696808 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.697432 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51818: BlockingRecv error: recv got EOF from 127.0.0.1:51818 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 18:45:10.706615 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.704476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.11.118.253:34717)
0929 18:45:10.704641 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.704648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.705377 (+ 729us) server_negotiation.cc:407] Connection header received
0929 18:45:10.705600 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.705605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.705669 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.705726 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.706489 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51820: BlockingRecv error: recv got EOF from 127.0.0.1:51820 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260929 18:45:10.718828 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.717438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.11.118.253:34717)
0929 18:45:10.717583 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.717588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.717775 (+ 187us) server_negotiation.cc:407] Connection header received
0929 18:45:10.717942 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.717946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.718009 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.718113 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.718704 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51836: BlockingRecv error: recv got EOF from 127.0.0.1:51836 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 18:45:10.738323 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.737004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.11.118.253:34717)
0929 18:45:10.737153 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.737158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.737293 (+ 135us) server_negotiation.cc:407] Connection header received
0929 18:45:10.737498 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.737501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.737554 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.737637 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.738179 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51838: BlockingRecv error: recv got EOF from 127.0.0.1:51838 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 18:45:10.774094 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.772993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.11.118.253:34717)
0929 18:45:10.773166 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.773169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.773278 (+ 109us) server_negotiation.cc:407] Connection header received
0929 18:45:10.773382 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.773384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.773437 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.773515 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.773986 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51854: BlockingRecv error: recv got EOF from 127.0.0.1:51854 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 18:45:10.841622 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.840352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.11.118.253:34717)
0929 18:45:10.840578 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.840581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.840622 (+ 41us) server_negotiation.cc:407] Connection header received
0929 18:45:10.840864 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.840867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.840921 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.841041 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.841522 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51866: BlockingRecv error: recv got EOF from 127.0.0.1:51866 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260929 18:45:10.973094 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:10.971936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.11.118.253:34717)
0929 18:45:10.972101 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:10.972105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:10.972252 (+ 147us) server_negotiation.cc:407] Connection header received
0929 18:45:10.972371 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:10.972373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:10.972421 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:10.972509 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:10.972932 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51880: BlockingRecv error: recv got EOF from 127.0.0.1:51880 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260929 18:45:11.232693 2069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 18:45:11.231550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.11.118.253:34717)
0929 18:45:11.231694 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.231697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.231814 (+ 117us) server_negotiation.cc:407] Connection header received
0929 18:45:11.231992 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.231996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.232056 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.232142 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 18:45:11.232593 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51882: BlockingRecv error: recv got EOF from 127.0.0.1:51882 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 18:45:11.497124 2157 catalog_manager.cc:1259] Loaded cluster ID: 87ce17bce65e478a9bf005cac95490c4
I20260929 18:45:11.497231 2157 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: loading cluster ID for follower catalog manager: success
I20260929 18:45:11.498968 2157 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: acquiring CA information for follower catalog manager: success
I20260929 18:45:11.500133 2157 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c503ee067d548669c96f1e13024a841: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:45:11.511731 2170 catalog_manager.cc:1259] Loaded cluster ID: 87ce17bce65e478a9bf005cac95490c4
I20260929 18:45:11.511857 2170 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: loading cluster ID for follower catalog manager: success
I20260929 18:45:11.513525 2170 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: acquiring CA information for follower catalog manager: success
I20260929 18:45:11.514791 2170 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d04b2238e2474d919b3cb4a7d6ce58c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 18:45:11.750886 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:11.747012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.11.118.253:34717)
0929 18:45:11.747257 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.747260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.747277 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:45:11.747398 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.747401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.747527 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.747638 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:11.747849 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.748397 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.749289 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.749519 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.749598 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:11.749968 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:11.750494 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:11.750590 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:11.750708 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 18:45:11.750760 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260929 18:45:11.758431 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:11.754465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.11.118.252:35085)
0929 18:45:11.754750 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.754755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.754772 (+ 17us) server_negotiation.cc:407] Connection header received
0929 18:45:11.754878 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.754880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.754996 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.755123 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:11.755353 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.755883 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.756739 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.756982 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.757095 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:11.757402 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:11.757992 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:11.758117 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:11.758157 (+ 40us) server_negotiation.cc:299] Negotiation successful
0929 18:45:11.758216 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":66,"threads_started":1}
I20260929 18:45:11.765621 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:11.761907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.11.118.252:35085)
0929 18:45:11.762023 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.762027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.762489 (+ 462us) server_negotiation.cc:407] Connection header received
0929 18:45:11.762578 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.762585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.762633 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.762687 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:11.762932 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.763434 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.764531 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.764756 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.764864 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:11.765162 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:11.765241 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:11.765310 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:11.765418 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 18:45:11.765475 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 18:45:11.766590 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:11.761611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.11.118.253:34717)
0929 18:45:11.761785 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.761789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.761905 (+ 116us) server_negotiation.cc:407] Connection header received
0929 18:45:11.762102 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.762105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.762170 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.762251 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:11.762803 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.763608 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.765095 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.765444 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.765538 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:11.765919 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:11.766007 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:11.766056 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:11.766226 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 18:45:11.766306 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 18:45:11.766773 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 18:45:11.761563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.11.118.254:44811)
0929 18:45:11.761928 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0929 18:45:11.761933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 18:45:11.761949 (+ 16us) server_negotiation.cc:407] Connection header received
0929 18:45:11.762048 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 18:45:11.762055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 18:45:11.762191 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 18:45:11.762398 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 18:45:11.762787 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.763653 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.765037 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 18:45:11.765347 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 18:45:11.765449 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 18:45:11.765909 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 18:45:11.766322 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 18:45:11.766410 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 18:45:11.766553 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 18:45:11.766624 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":63,"threads_started":1}
I20260929 18:45:11.770825 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1921
I20260929 18:45:11.777061 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 1996
I20260929 18:45:11.783066 11739 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqumnYE/build/debug/bin/kudu with pid 2070
2026-09-29T18:45:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3184 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6310 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288618 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST